Spectral imaging system for remote and noninvasive detection of target substances using spectral filter arrays and image capture arrays

ABSTRACT

An approach to noninvasively and remotely detect the presence, location, and/or quantity of a target substance in a scene via a spectral imaging system comprising a spectral filter array and image capture array. For a chosen target substance, a spectral filter array is provided that is sensitive to selected wavelengths characterizing the electromagnetic spectrum of the target substance. Elements of the image capture array are optically aligned with elements of the spectral filter array to simultaneously capture spectrally filtered images. These filtered images identify the spectrum of the target substance. Program instructions analyze the acquired images to compute information about the target substance throughout the scene. A color-coded output image may be displayed on a smartphone or computing device to indicate spatial and quantitative information about the detected target substance. The system desirably includes a library of interchangeable spectral filter arrays, each sensitive to one or more target substances.

PRIORITY

This application is a continuation of U.S. patent application Ser. No.15/379,059, filed Dec. 14, 2016, which is a divisional of U.S. patentapplication Ser. No. 14/742,074 filed Jun. 17, 2015, now U.S. Pat. No.9,551,616, which claims priority under 35 U.S.C. § 119(e) from UnitedStates Provisional patent application having Ser. No. 62/014,004, filedon Jun. 18, 2014, by Gary L. McQuilkin and Gregory L. Engelke, andtitled SPECTRAL IMAGING SYSTEM FOR REMOTE AND NONINVASIVE DETECTION OFTARGET SUBSTANCES USING SPECTRAL FILTER ARRAYS AND IMAGE CAPTURE ARRAYS,wherein the entireties of which are respectively incorporated herein byreference for all purposes.

FIELD OF THE INVENTION

This invention relates to using spectral imaging analysis tononinvasively and remotely detect the presence, location, and/orquantity of a target substance in a scene. More specifically, thisinvention relates to using spectral imaging analysis to noninvasivelyand remotely detect the presence, location, and/or quantity of a chosentarget substance via a spectral imaging system comprising a spectralfilter array and an image capture array.

BACKGROUND OF THE INVENTION

Spectral imaging is a branch of spectroscopy and of photography in whichat least some spectral information is collected from an image plane(e.g., two-dimensional image) for a scene of interest. An image capturedevice may be aimed at a scene in order to capture image information forthat scene. A variety of spectral imaging methodologies are known.Examples include hyperspectral imaging, multispectral imaging (a type ofhyperspectral imaging), full spectral imaging, imaging spectroscopy,chemical imaging, and the like. Historically, hyperspectral imaging andmulti-spectral image analysis have been associated with satellite,airborne, or large scale operations using large, expensive camerasystems that are not well-suited for handheld operation or routinebusiness and consumer applications.

Spectral imaging generally involves capturing spectral information fromone or more portions of the electromagnetic spectrum. Although spectralinformation for any wavelengths in the electromagnetic spectrum may beused, often spectroscopy uses spectral information for wavelengths inthe range from about 100 nm to about 14,000 nm. For reference, it isoften convenient to divide the span of the electromagnetic spectrum intothe following bands: ultraviolet (UV) band from 100 nm to 400 nm;visible (VIS) band from 400 to 700 nm; near infrared (NIR) band from 700to 1500 nm; short-wave infrared (SWIR) band from 1500 to 3000 nm;mid-wave infrared (MWIR) band from 3000 to 5000 nm; and long-waveinfrared (LWIR) band from 5000 to 14000 nm. The ultraviolet bandincludes the following sub-bands: far ultraviolet (FUV) band from 122 to200 nm; middle ultraviolet (MUV) band from 200 to 300 nm; and nearultraviolet (NUV) band from 300 to 400 nm. The ultraviolet band also isdivided into the following sub-bands: ultraviolet C (UVC) band from 100to 280 nm; ultraviolet B (UVB) band from 280 to 315 nm; and ultravioletA (UVA) band from 315 to 400 nm.

Spectral information captured for a scene may be represented as an imagecube, which is a type of data cube. A data cube generally is a threedimensional array of data values. In spectroscopy, one kind of data cuberesults when a spectrally-resolved image is represented as a threedimensional volume in which the captured image is represented in a twodimensional image while spectral information associated with individualpixels, or groups of pixels, is incorporated into at least a thirddimension of the data cube.

Conventional hyperspectral imaging is a powerful but expensive analysistechnology for remotely determining the chemical composition of asurface. For example, hyperspectral imaging typically generates a datacube with two spatial dimensions and one spectral dimension. The twospatial dimensions correspond to the spatial data that might berepresented in a common digital photograph. However, each spatial pixelin a hyperspectral data cube also is associated with an electromagneticspectral array spanning wavelengths that often extend well beyond thevisible spectrum to higher and/or lower wavelengths.

A conventional hyperspectral imaging system scans a scene to captureimage information. Line scanning often is used. For example, anillustrative, conventional hyperspectral system has a linear array of256 imaging elements. This linear array is scanned across the targetsurface in a line-scan or “push-broom” format. In this manner the lineararray can generate a three-dimensional data cube with two spatialdimensions and one spectral dimension. A typical data cube may havedimensions of 256×256 spatial dimensions by 320 spectral bins. Spatiallines or “frames” are acquired sequentially at a frame rate of 100 to400 frames per second. This line scan modality, acquires complete imageinformation over a second or longer. This makes it difficult to acquirehigh resolution images for moving subjects.

Although conventional hyperspectral imaging provides powerful analysiscapabilities, additional significant limitations exist for anywidespread application of this technology as conventionally practiced.Foremost is a high price of $100,000 to $200,000 per system. The lineardetection array imposes a scanning modality requiring a precisionfixture with a uniform scanning speed between target surface and lineardetection array. The line scan requirement also eliminates thepossibility of acquiring a truly simultaneous image since the detectorscans across the target surface in sequential lines. Undue movement ofthe target with regard to the scanning coordinates tends to result in anunrecognizable spatial shape in the resulting image. With movement, theresulting spectral information for a given pixel also may be distorted.The scanning requirement, alignment complexity and necessary fixturingoften results in a stationary system that is not easily transported.With a high price tag and precise scanning restrictions, conventionalhyperspectral imaging systems are limited primarily to inspectionsystems in large scale operations for high-volume production lines suchas food processing, garbage sorting, or mineral analysis. Conventionalhyperspectral imaging is not well suited for mobile applications,high-resolution systems, multi-line operation, bench-scale laboratoryanalysis, small business or consumer uses.

Multispectral imaging also offers the potential for remotely determiningthe chemical composition of a surface. Similar to a hyperspectral image,a multispectral image contains data with both spatial and spectraldimensions. Unlike the hyperspectral image, which contains a fullspectrum for each spatial pixel, a multispectral image contains a fixednumber of broad spectral bands for each spatial pixel. Conventionalmultispectral imaging systems typically are designed for one or morespecific initial uses but are difficult to reconfigure for other uses.

The digital photography and digital camera industries have developedhigh quality, low cost sensors that capture images for portions of theelectromagnetic spectrum that fall within the visible spectrum, namely,400 nm to 700 nm. Both professional and consumer markets show stronginterest in and benefit from these technologies. Digital cameras andphones typically use either charge-coupled device (CCD) or CMOS imagesensors to capture images. These CCD and CMOS sensors can be obtainedwith a wide range of resolutions, e.g., from 540×720 (0.39 MP), to5184×3456 (18 MP) in a number of digital, single lens reflex cameras(Canon EOS 60D Digital SLR Camera with lens kit, 5184×3456 pixels, 18MP, $1300) and lower cost, compact cameras (Canon PowerShot, A1300, 16MP, $119). Recent advances have led to miniaturization such that thesecameras are routinely integrated into small form factors as thin as 7.6mm (0.3 in.), including the lens. An example of a product thatintegrates such a miniaturized form factor are the iPhone 5 smartphoneavailable from Apple Inc. as well as other smartphones. In addition tosmartphone technology, there are a number of other mobile and economicalcomputing technologies such as tablet, mini-tablet, laptop, and desktoptechnologies. Each of these alternative technologies shares manycapabilities with those of smartphones.

Smartphone and other mobile computing technologies such as touchsensitive tablets have significant computing and connection power aswell as the inclusion of digital cameras. While early models had onlylimited photography capability, the latest smartphones, such as theiPhone 5 and Samsung Galaxy S III have dual digital cameras (front andback facing) with resolutions of 8 MP/1.2 MP and 8 MP/1.9 MP,respectively. In addition to camera features, these smartphones have thecapability for GPS location and navigation, as well as sensing verticaland horizontal phone orientation. Communication capabilities includeBluetooth and Wi-Fi standards.

Researchers at the University of Illinois are developing asmartphone-based spectrometer. See Liz Ahlberg, Cradle Turns Smartphoneinto Handheld Biosensor, University of Illinois, News Bureau, PublicAffairs, May 23, 2013. A custom cradle holds the smartphone in fixedalignment with optical components that include a photonic crystalbiosensor. This device detects shifts in the resonant wavelength of thebiosensor on the order of 5 nm. The target must be dissolved in a smallvial of liquid and placed on a microscope slide. The slide is in turnplaced in a slot on the cradle attached to the smartphone. While thisdevice contains a smartphone display for observing a single, resonantwavelength of a biosensor for the given target substance on themicroscope slide, it does not provide mobile, high-resolution, chemicalimaging capability. This device cannot be used for high volume orinstantaneous analysis of a target substance due to the time required todissolve the target substance in water, place the solution on amicroscope slide and await a 30 second analysis which consults aweb-based, data base.

Heinold, U.S. Pat. Application No. 20140022381, describes a spectralimaging apparatus that includes a multi-camera system having multiplecamera elements, a set of filter elements attached to the cameraelements and a light-sensor array, and a second set of filter elementsattached to the light-sensor elements.

Even with the multitude of technologies described above, there presentlyexists no mobile, economical and convenient method or apparatus torapidly, remotely and accurately detect, locate or quantify informationof interest, such as the presence or amount of a target substance atparticular location(s) in a scene. Therefore, there is a strong need fora spectral imaging system that has detection capabilities associatedwith hyperspectral and multispectral imaging systems but is economical,able to acquire high-resolution data rapidly from moving targets, ismobile, and is suitable for agricultural, medical, veterinary,sanitation, industrial, business, and consumer uses. Also, there is astrong need for spectral imaging systems whose detection capabilitiescan be easily changed on demand to be able to detect information for awide range of desired applications.

SUMMARY OF THE INVENTION

The present invention relates to spectral imaging systems and methodsfor noninvasively and remotely detecting the presence, location, and/orquantity of a target substance in a scene of interest. Morespecifically, this invention relates to using spectral imaging analysisto noninvasively and remotely detect the presence, location, and/orquantity of a target substance via a spectral imaging system comprisinga spectral filter array and an image capture array. The spectral filterarray comprises a plurality of filter elements each of which selectivelypasses a selected bandwidth portion of the electromagnetic spectrum(e.g., one or more specific wavelengths or relatively narrow range ofwavelengths) that help to characterize the spectrum of the chosen targetsubstance. By appropriate selection of the filter elements used in thespectral filter array, the spectral filter array can be customized toinclude a sufficient number of filter elements that allow that filterarray to be used to detect a target substance of interest. In this way,each spectral filter array is associated with at least one specifictarget substance and may be used to detect the associated targetsubstance.

The system desirably includes a plurality of interchangeable spectralfilter arrays (also referred to herein as a library of spectral filterarrays) that interchangeably align with the image capture array ondemand. This allows the system to be used to detect a wide range oftarget substances. For example, the system is easily configured todetect a particular target substance by selecting and deploying thecorresponding spectral filter array in optical alignment with the imagecapture array. The system can be easily re-configured to detect anothertarget substance by selecting and deploying an alternative filter arrayand implementing an analysis algorithm for such other target substanceor by using a different analysis algorithm associated with the othertarget substance but using the same filter array. The spectral filterarray may be changed via a simple coupling mechanism such as by placinga different spectral filter array module in an attachment holder, suchas a filter slot, clip, spring-loaded attachment, screws, bolts, snapfit engagement, adhesive, or hook and loop fastener (e.g., the Velcrobrand hook and loop fastener), compartment, retainer, spring,combinations of these, and the like. The spectral filter array may alsobe changed electronically such as by electronic control of anelectronically tunable filter array. In the sense of being quicklyconfigured for multiple detection applications, the present inventionprovides a “universal” detection system.

The image capture array comprises a plurality of image capturingelements that are aligned with corresponding filter element(s) in amanner effective to capture a plurality of spectrally filtered imagesfor the scene of interest. The captured image information is analyzed todetermine information indicative of the presence, location, and/orquantity of the target substance in the scene. For example, individualpixels, or groups of pixels, may be analyzed to determine if thepixel(s) are associated with spectral responses matching the spectrum ofthe target substance. If a match is found, the target substance isdetected and the location of the pixel(s) in the image information helpsto precisely locate the target substance in the scene that was imaged.In embodiments that use high resolution image capturing elements, thisallows even minute traces of a target substance to be detected if evenonly a single pixel in an image corresponds to the target substance.Analysis of the image information can also be used, as described furtherbelow, to quantify the amount of target substance that is detected. Thepresence, location, and/or quantity of the target substance are computedand may be displayed on an output image. For example, the system canoutput an image in which image pixels associated with the targetsubstance are highlighted in some way to show location and even quantityof the target substance. This invention provides a significantimprovement over conventional hyperspectral imaging and multi-spectralimaging systems in the areas of cost, resolution, simultaneous imageacquisition, imaging of moving targets, miniaturization, mobileoperation, and ease of adapting to different applications.

The present invention further provides methods to select the filterelements that are used in a particular spectral filter array to allowthe system to detect a target substance of interest. According to oneapproach for selecting such filter elements, the spectrum of the targetsubstance across a broad range of wavelengths is provided. In manyembodiments, the spectrum is obtained over a relatively broad wavelengthrange spanning at least 14,000 nm, or even at least 5000 nm, or even atleast 2000 nm, or even at least 1000 nm, or even at least 500 nm, oreven at least 300 nm, or even at least 200 nm, or even at least 100 nm.For example, in many modes of practice, such as when obtaining aspectrum over a typical range of CCD sensor sensitivity, a span of 1000nm from about 200 nm to about 1200 nm would be suitable. To helpdistinguish the target substance from other materials that might be inthe same scene, spectra for one or more background substances over asimilar wavelength range also may be provided. The spectra of the chosentarget substance and anticipated background substance(s) are analyzedand specific bandwidth portions (e.g., within the wavelength range of200 nm to 1200 nm, wavelength bandwidths of up to 30 nm, or up to 20 nm,or up to 15 nm, or up to 10 nm, or up to 5 nm, or even specificwavelengths in many modes of practice) of the target spectrum areselected that identify and differentiate the target substance spectrumfrom the spectra of anticipated background substances. Filter elementsthat selectively and respectively pass the selected bandwidth portionsare provided and incorporated into a spectral filter array.

As used herein, bandwidth for a filter refers to the difference betweenthe upper and lower cutoff wavelengths. A cutoff wavelength is aboundary in a filter's spectral response at which the transmissionamplitude is reduced to 50% of the maximum transmission amplitude. Inactual practice, a filter element used in the practice of the presentinvention may not attenuate all wavelengths outside the desiredwavelength range completely. Instead, there often is a region justoutside the intended passband where wavelengths are attenuated, but notfully rejected. For purposes of the present invention, and recognizingsuch attenuation, the bandwidth of an optical filter element is deemedto be the full width half maximum (FWHM) bandwidth. Thus, the bandwidthspecification may be designated as the width of the spectral curve forwavelengths that are half of the maximum transmission amplitude. Anexample of specifications for a spectral filter element may be asfollows: center wavelength, 980+/−2.00 nm; minimum transmission, ≥85%;full-width, half maximum (FWHM), 10.00+/−2.00 nm; blocking wavelengthrange, 200-1200 nm; optical density, 4.

The image capture array is aligned with the spectral filter array sothat image capture elements of the image capture array can capture afiltered image through at least one corresponding filter element.Depending upon the target substance at issue, all or only a portion ofthe filter elements in a spectral filter array may be used to capturefiltered images. After an array of filtered images is captured, thefiltered images are typically stored in electronic memory and processed,either in real-time or otherwise. The system uses the captured, filteredimage information to determine which portion(s) of the captured imageinformation (if any) have spectral characteristics that match thespectral characteristics of the target substance.

The analysis of captured image information may be performed by anysuitable computing device. Suitable mobile platforms include asmartphone, laptop computer, tablet computer, mini-tablet computer,desktop computer, head-mounted computer, mobile computer, or cloud-basedcomputer.

In illustrative modes of practice, program instructions analyze thespectrally filtered images via a target substance algorithm that iscapable of analyzing image information to determine which portion(s), ifany, of the image information have spectral characteristics that matchthe spectral characteristics of the target substance. For example,individual pixels, or groups of pixels, of captured image informationcan be analyzed to assess if the pixel(s) display spectralcharacteristics of the target substance. This analysis may be used todetermine the presence, location, and/or quantity of the chosen targetsubstance within the scene of interest. Additionally, the programinstructions may generate an output image that displays the presence,location and/or quantity of the target substance with respect to objectswithin the scene.

The spectral imaging system of the present invention may be calibratedin a variety of ways to enhance the performance of image capture andanalysis. For example, in many modes of practice, calibration occurs byusing either an in-frame reference or an accumulation of nominalcomponent specifications. The in-frame reference, having known spectralcharacteristics on its surface, may be used to account for variations inillumination, filter element attenuation, and image sensor sensitivitytogether as an assembled system for each corresponding elemental pairwithin the spectral filter array and image capture array. Additionally,a cumulative calibration for each corresponding elemental pair may becomputed by using the manufacturing specification for each component.The required accuracy and system cost goals for a given detection of achosen target substance will determine which calibration method isadvantageous, with a quantity indicator likely to require the moreaccurate, in-frame reference calibration. Because the system uses anarray of image capture elements to capture an array of filtered images,the individual images may be aligned (also referred to in the imagingindustry as image registration) so that features in one image areaccurately matched and aligned with the same features in the otherimage(s).

In many modes of practice, the present invention satisfies the desire tobe able to detect a target substance remotely and noninvasively. Thisavoids the time and cost associated with a laboratory analysis. Using anappropriate spectral filter array according to the principles of thepresent invention, the system may be used to detect a target substancethat is solid, liquid, gas or plasma. The target substance may be a purechemical substance, a compound of chemical substances, or a mixture ofchemical substances. The present invention can be used to detect anykind of target substance that has spectral characteristics that can beviewed within at least portions of the fields of view of the imagecapture array. Individual pixels, or groups of pixels, of captured imageinformation can be analyzed to assess if the pixel(s) display spectralcharacteristics of the target substance. In embodiments that use highresolution image capturing elements, this allows even minute traces of atarget substance to be detected if even only a single pixel in an imagecorresponds to the target substance. Moreover, by recognizing whichpixel(s) of captured image information correspond to a target substance,the precise location(s) of a target substance in an image can be locatedand identified.

In representative modes of practice, this invention provides thecapability to provide a powerful chemical imaging tool, sensitive to awide range of spectral wavelengths, in a package that can be small,light-weight, battery-powered, and mobile. The output can be providedrapidly without the delay associated with a laboratory analysis. Thespatial location of a target substance within the field of view may beshown on a representative image of the scene. For example, the systemcan output an image in which image pixels associated with the targetsubstance are highlighted in some way to show location and even quantityof the target substance. If desired, spectral imaging systems also maybe provided as a stationary system with a cost that is a small fractionof the cost commonly associated with systems providing similar chemicalimaging capability. For instance, one representative implementation ofthis invention is a handheld, multi-camera adapter integrated with asmartphone as illustrated in FIGS. 5A, 5B and 5C. The multi-cameraadapter includes multiple camera elements as an image capture array, andthese camera elements are optically aligned with the filtering elementsof the spectral filter array incorporated into an interchangeable,spectral filter card. The system may be re-configured to be sensitive toa different target substance by choosing a different spectral filtercard from a library of spectral filter cards, each card including anarray of filter elements that is designed to capture identifyingspectral information for specific target substance(s). In some modes ofpractice, the system may be re-configured to be sensitive to a differenttarget substance by selecting a different target substance algorithm tobe used with the same filter card. The system would be capable ofdetecting a vast number of different target substances determined by thefilter card selection. The cost of the multi-camera adapter and spectralfilter card is a small fraction of the cost necessary to achieve similarchemical imaging capability with conventional hyperspectral imaging orother remote sensing systems. Additionally, the image resolutionavailable with the present invention may be 10 to 100 times greater thanthe more expensive, conventional systems.

Advantageously, the present invention provides a high level ofperformance using a wide variety of low-cost, digital camera sensors.Many of these sensors have a native spectral sensitivity of 300 nm to1100 nm (prior to the addition of filters intended to reduce sensitivityto only the visible spectrum, 400-700 nm) and may be used to provide ahigh-resolution, low-cost, multi-camera array capable of acquiringmultiple images simultaneously. This permits simultaneous imageacquisition, which is very desirable for high-speed operation or movingsubjects. Such simultaneous image acquisition for moving subjects is notpossible with conventional hyperspectral imaging systems that rely online-scan image acquisition. In addition, the optically aligned filterelements, which determine the wavelength sensitivity for each cameraelement, may be mounted on a filter card, permitting different targetsubstances to be selected simply by changing filter cards and internalsoftware algorithms.

In a typical mode of practice, spectral information is acquired for adesired target substance. This spectral information includes spectralinformation for the target substance and may, if desired, includespectral information for anticipated background substances. The spectralinformation preferably is analyzed to determine a finite number ofspecific wavelengths that in combination uniquely distinguish thespectral characteristics of the target substance from the spectra of oneor more background substances. These pre-determined wavelengths andknowledge of the spectral characteristics of the target substance andoptionally the background are then used to create a spectral imagingsystem. The system includes an image capture array (also referred toherein as a multi-camera array) having a plurality of image capturingelements (which in some embodiments are individual camera elements) thatare sensitive to a unique bandwidth portion of the electromagneticspectrum. The unique spectral sensitivity for each image capturingelement of the multi-camera array is achieved via a filter arrayincluding a plurality of filter elements. Each filter element in thefilter array is optically aligned with a corresponding image capturingelement of the multi-camera array. The filter characteristics of eachfilter element are selected so that each filter selectively passes abandwidth portion that encompasses a pre-determined wavelength.Preferably, the spectral filter array is interchangeable with otherfilter arrays so that the detection capabilities of the system can bequickly configured to detect any of a wide variety of target substancesby selection of an appropriate spectral filter array.

In some embodiments the spectral filter array may even be morepermanently secured to the imaging system if desired. This may bedesired where the system will be used in rugged or dangerousenvironments to help prevent a spectral filter array from being easilydislodged. Examples of more secure fastening include screws, bolts,welds, fusing, adhesives, rivets, clamps, wiring, combinations of these,and the like.

For example, in one illustrative mode of practice, six wavelengths areselected that allow a target substance to be selected from a backgroundsubstance. An image capture array with six or more image capturingelements may be fitted with a filter array including six filteringelements. Each of the filtering elements is designed to selectively passa bandwidth portion encompassing one of the selected wavelengths orwavelength bandwidths. The imaging system then may capture sixcorresponding, filtered images. In many modes of practice, capturedimages, each representing a unique portion of the spectrum, optionallyare processed to spatially align pixels in each image such that theyrepresent the same spatial point on the image plane within the field ofview of the system.

Capturing simultaneous images allows analysis of moving objects withoutsuffering from motion distortion and blurring. Additionally, thesimultaneous acquisition of images from multiple camera elements enablesa solid-state system that does not require a mechanically rotatingfilter wheel as is often used with systems that implement sequentialfiltering methods. Additionally, the simultaneous acquisition of imagespermits high quality output images when processing is conducted at avideo rate or higher.

The system then analyzes the captured image information to determinewhich pixels of the image information display spectral characteristicsof the target substance. For instance, the spectrally-distinct,spatially aligned images may be processed to mathematically deriveinformation regarding the presence and quantity of the target substanceat points on the target surface or within the target volume, optionallyaccounting for variations in spectral illumination, camera elementsensitivity and filter element attenuation. An output, such as an outputimage, may be computed that overlays information about the position andquantity of the target substance on a spatial orientation image (SOI)representing the spatial position of objects within the field of view.This output image may provide information about only the presence of thetarget substance at each spatial position, or it may also provideinformation about the quantity of the target substance at each spatialposition.

In other modes of practice, the arrays may include more or less than sixelements. In some modes of practice, an array may include a greaternumber of array elements, e.g., ten or more elements, but only some ofthese elements, e.g., 3 or 4 elements, are needed and used to captureinformation for a particular target substance. In other modes ofpractice, a single bandwidth portion may be suitable to encompass two ormore pre-determined wavelengths. In such embodiments, a single imagecapturing element and a single corresponding filtering element may beused to capture spectral information for such two or more pre-determinedwavelengths.

In one aspect the present invention is connected to any number ofoutside devices via connectivity standards common to smartphones andcomputing devices. Examples of such connectivity interfaces include, butare not limited to, Wi-Fi, Bluetooth, and/or numerous additionalwireless standards. In another embodiment, the wireless link may bereplaced by any suitable wired link. Examples of wired links areselected from at least one of USB, Firewire, Ethernet, custom,proprietary, or a multitude of other wired standards.

In one aspect, the present invention relates to a spectral imagingsystem for remotely and noninvasively detecting the presence andlocation of one or more target substances in a scene, said systemcomprising:

-   -   a) an image capture array comprising a plurality of image        capturing elements;    -   b) a plurality of spectral filter arrays, wherein:        -   (i) each spectral filter array is pre-associated with at            least one corresponding target substance;        -   (ii) each spectral filter array includes a plurality of            spectral filtering elements, wherein each spectral filtering            element of said plurality of spectral filtering elements            selectively passes a pre-selected bandwidth portion of the            electromagnetic spectrum that is pre-associated with a            spectral characteristic of the corresponding target            substance;        -   (iii) each of the plurality of spectral filter arrays is            selectable and positionable on demand to be aligned with the            image capture array such that each of at least two or more            of the image capturing elements is optically aligned with at            least one corresponding spectral filtering element in a            manner effective to capture spectrally filtered image            information comprising image pixels; and    -   c) program instructions that use the spectrally filtered image        information to determine if image pixels are associated with        spectral responses matching the target substance and to        determine and output information indicative of the presence and        location of the at least one associated target substance in the        scene.

In another aspect, the present invention relates to a method of remotelyand noninvasively detecting at least one target substance in a scene,comprising the steps of:

-   -   a) providing an image capture array comprising a plurality of        image capturing elements;    -   b) providing a spectral filter array, wherein:        -   (i) the spectral filter array is associated with at least            one corresponding target sub stance;        -   (ii) the spectral filter array includes a plurality of            spectral filtering elements, wherein at least two spectral            filtering elements of said plurality each selectively passes            a pre-selected bandwidth portion of the electromagnetic            spectrum that is pre-associated with a spectral            characteristic of the corresponding target substance;        -   (iii) the spectral filter array is aligned with the image            capture array such that each of at least two or more of the            image capturing elements is optically aligned with at least            one corresponding spectral filtering element in a manner            effective to capture spectrally filtered image information            comprising image pixels;    -   c) aiming the image capturing elements of the image capture        array at the scene;    -   d) using the image capture array and the spectral filter array        to capture a plurality of filtered images of the scene; and    -   e) using information comprising the filtered, captured images to        determine if image pixels are associated with spectral responses        matching the target substance and to determine and output        information indicative of the presence and location of the at        least one associated target substance in the scene.

In another aspect, the present invention relates to a method ofproviding a spectral imaging system, comprising the steps of:

-   -   a) providing spectral information for a target substance;    -   b) using the spectral information to associate at least first        and second bandwidth portions of the electromagnetic spectrum        with spectral characteristics of the target substance;    -   c) providing a spectral filter array comprising a plurality of        spectral filtering elements, wherein a first spectral filtering        element selectively passes the first bandwidth portion of the        electromagnetic spectrum and a second spectral filtering element        selectively passes a second bandwidth portion of the        electromagnetic spectrum; and    -   d) causing an image capture array comprising a plurality of        image capturing elements to be aligned with the spectral filter        array in a manner effective to capture first and second,        filtered images through the first and second spectral filtering        elements in a manner such that image pixels of the filtered        images that match the spectral characteristics of the target        substance indicate the presence and location of the target        substance in a scene.

In another aspect, the present invention relates to a method ofprocessing image information, comprising the steps of:

-   -   a) capturing a first filtered, spectral image of at least a        portion of a scene wherein the first filtered spectral image is        obtained using a first spectral filter element that selectively        transmits a first bandwidth portion of the electromagnetic        spectrum within which a target substance has a first        pre-determined spectral response;    -   b) simultaneously capturing a second filtered, spectral image of        at least a portion of the scene, wherein the second filtered        spectral image is obtained using a second spectral filter        element that selectively transmits a second bandwidth portion of        the electromagnetic spectrum within which the target substance        has a second pre-determined spectral response, and wherein the        first and second filtered, spectral images comprise image        pixels; and    -   c) using information comprising the first and second filtered,        spectral images to determine if image pixels are associated with        spectral responses matching the target substance and to        determine and output information indicative of the presence and        location of the at least one associated target substance in the        scene.

In another aspect, the present invention relates to a spectral imagingsystem, comprising:

-   -   a) an image capture array comprising a plurality of image        capturing elements;    -   b) a plurality of spectral filter arrays, wherein:        -   (i) each spectral filter array is pre-associated with at            least one corresponding target substance; and        -   (ii) each spectral filter array includes a plurality of            spectral filtering elements, wherein each spectral filtering            element of said plurality of spectral filtering elements            selectively passes a pre-selected bandwidth portion of the            electromagnetic spectrum that is pre-associated with a            spectral characteristic of the corresponding target sub            stance;        -   (iii) each of the plurality of spectral filter arrays is            selectable and positionable on demand to be aligned with the            image capture array such that each of at least two or more            of the image capturing elements is optically aligned with at            least one corresponding spectral filtering element in a            manner effective to capture spectrally filtered image            information in a manner such that image pixels of the            filtered images that match the spectral characteristics of            the target substance indicate the presence and location of            the target substance in a scene.

In another aspect, the present invention relates to a method ofproviding an imaging system for detecting a target substance in a scene,comprising the steps of:

-   -   a) providing spectral information for a target substance;    -   b) using the spectral information to associate a plurality of        specific bandwidth portions of the electromagnetic spectrum with        spectral characteristics of the target substance;    -   c) using the selected bandwidth portions to provide a spectral        imaging system comprising:        -   i) an image capture array comprising at least two image            capturing elements, each image capturing element capable of            independently capturing an image of the scene;        -   ii) a filter array comprising a plurality of filter            elements, each sensitive to a pre-determined specific            bandwidth portion, and wherein the filter elements are each            aligned with a corresponding camera element of the image            capture array in a manner such that the image capture array            captures image information for a plurality of independent            spectrally filtered images;        -   iii) program instructions that use the captured image            information to identify at least one spatial location of the            target substance in the scene.

In another aspect, the present invention relates to an imaging systemfor detecting a target substance within a scene, comprising:

-   -   a) an image capture array with at least two image capturing        elements, each image capturing element capable of independently        capturing an image of a scene;    -   b) a spectral filter array comprising a plurality of spectral        filtering elements that are each optically aligned with a        corresponding image capturing element of the image capture array        in a manner such that the image capturing array captures a        plurality of independently, selectively, and spectrally filtered        images, and wherein each spectral filtering element of said        plurality of spectral filtering elements selectively passes a        pre-selected bandwidth portion of the electromagnetic spectrum        that is pre-associated with a spectral characteristic of the        corresponding target substance;    -   c) program instructions that use a plurality of captured images        to provide an output indicative of the presence and location of        the target substance is in the scene.

In another aspect, the present invention relates to a method ofdetecting a target substance in a scene, comprising the steps of:

-   -   a) using an image capture array and a spectral filter array, to        simultaneously capture a plurality of spectrally filtered images        of the scene, wherein the spectral filter array is sensitive to        at least two bandwidth portions of the electromagnetic spectrum        which provide spectral information effective to identify the        target substance; and    -   b) using the spectrally filtered images to generate an output        indicative of the presence and the one or more locations of the        target substance in the scene.

In another aspect, the present invention relates to a spectral filtersystem, comprising a plurality spectral filter arrays, wherein eachspectral filter array comprises a plurality of spectral filteringelements, and wherein each spectral filter array is pre-associated withat least one target substance, and wherein each spectral filteringelement of said plurality of spectral filtering elements selectivelypasses a pre-selected bandwidth portion of the electromagnetic spectrumthat is pre-associated with a spectral characteristic of thecorresponding target substance.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 schematically shows a block diagram of set-up and operation of asystem for detecting a target substance using principles of the presentinvention.

FIG. 2A shows absorption spectral information for water between 400 nmand 1100 nm.

FIG. 2B shows sample reflectance spectral information for green grass,dry grass and soil.

FIG. 2C shows absorption spectral information of hemoglobin andoxyhemoglobin between 600 nm and 1000 nm.

FIG. 2D shows absorption spectral information of urine between 200 nmand 550 nm.

FIG. 2E shows absorption spectral information for normal tissue and atumor between 650 nm and 1050 nm.

FIG. 3A shows the differentiation of spectra for water and backgroundsubstances using two wavelengths.

FIG. 3B shows the differentiation of spectra for durum wheat, barley andtriticale using six wavelengths.

FIG. 4 shows how a spectral imaging system embodiment of the presentinvention captures image information at selected wavelengths and usesthat captured image information to detect the presence and location of atarget substance in a scene.

FIG. 5A shows a rear, perspective view of an embodiment of the presentinvention in the form of a spectral imaging system incorporating a smartphone including an image capture array with an interchangeable spectralfilter card and a spectral filter array library.

FIG. 5B shows a front perspective view of the spectral imaging system ofFIG. 5A.

FIG. 5C shows a filter card library included in the spectral imagingsystem of FIG. 5A.

FIG. 6 shows an alternative embodiment of a spectral imaging system ofthe present invention.

FIG. 7 shows an alternative embodiment of a spectral imaging system ofthe present invention.

FIG. 8 shows an alternative embodiment of a spectral imaging system ofthe present invention.

FIG. 9 shows an alternative embodiment of a spectral imaging system ofthe present invention.

FIG. 10 shows an embodiment of the present invention using two differentspectral imaging systems to observe both sides of a target.

FIG. 11 shows an embodiment of a camera element of the presentinvention.

FIG. 12 shows an embodiment of a six-element, multi-camera array of thepresent invention.

FIG. 13 shows an embodiment of a spectral imaging system of the presentinvention with an interchangeable filter array.

FIG. 14 shows an embodiment of a spectral imaging system of the presentinvention with an interchangeable filter array.

FIG. 15 shows an embodiment of a spectral imaging system of the presentinvention analyzing a crop.

FIG. 16 shows spectral information illustrating selected wavelengthsuseful in the detection and analysis of a vegetation target substance.

FIG. 17 shows a schematic of an output image of a human retina withdifferentiated arterioles and venules achieved due to the differences inspectral characteristics of oxyhemoglobin and deoxyhemoglobin that allowprinciples of the present invention to detect arterial/venous blood andmeasure oxygen saturation in a tissue.

FIG. 18 shows how the spectral information for organic contaminants aredistinguished from the spectral information of a stainless steelsurface.

FIG. 19A shows a schematic image of a contaminated, stainless steelsurface.

FIG. 19B shows how the present invention can output an image showing thepresence and location of contaminants on the stainless steel surface ofFIG. 19A.

FIG. 20 schematically shows output images of poultry carcasses in whichprinciples of the present invention are used to detect contaminants onthe carcasses.

FIG. 21 shows a mirror configuration of the present invention using atwo-element, image capture array.

FIG. 22A shows a mirror configuration of the present invention using athree element image capture array.

FIG. 22B shows a mirror configuration of the present invention using afour element image capture array.

FIG. 23 shows a mirror configuration of the present invention using asix-element image capture array.

FIG. 24 shows a two-element mirror configuration of the presentinvention using off-axis parabolic mirrors.

FIG. 25A is a ‘full spectrum’ image taken with the narrower visiblespectrum filter removed (typically included with digital cameras).

FIG. 25B is an NIR detection image, sensitive to a narrow bandwidthcentered on the absorption peak of water at 980 nm.

FIG. 25C is a spectral reference image centered in an area of thespectrum which exhibits little spectral absorption for water at 766 nm.

FIG. 25D is a ratio image representing the reference image divided bythe NIR image.

FIG. 25E is an output image created from the reference image, showingregions of the ratio image where the water is present and highlightingthe position of water via a custom colormap.

FIG. 26 shows a colormap index to highlight the presence of water in theoutput images shown in FIG. 25E.

FIG. 27 shows an output image that highlights the presence of water in ascene.

FIG. 28A shows spectral information for barley, durum wheat, andtriticale.

FIG. 28B shows an illustrative colormap used to generate the outputimage of FIG. 28C.

FIG. 28C shows how the spectral information of FIG. 28A and the colormapof FIG. 28B is used to detect barley in a scene relative to durum wheatand triticale.

FIG. 29A shows spectral information for egg conditions.

FIG. 29B shows first derivative spectral information for the spectralinformation of FIG. 29A.

FIG. 29C shows a plot of amplitudes of the spectral information in FIG.29A for different egg conditions.

FIG. 30 shows an example algorithm for identifying spectral wavelengthsto use for detection of the egg conditions whose spectral information isshown in FIG. 29A.

FIG. 31 shows absorption spectra of oxyhemoglobin and deoxyhemoglobin.

FIG. 32A shows spectral information for road conditions.

FIG. 32B shows how spectral information can be used to detect and locateroad conditions in a road scene as viewed from the perspective of atraffic camera.

FIG. 33 shows how spectral information can be used to detect and locateroad conditions in a road scene as viewed from the perspective of avehicle.

FIG. 34A shows a visible color image (VIS-COLOR, 400-700 nm) of a sceneincluding several liquids held in various containers.

FIG. 34B shows a reference image (REF, 670 nm) of the scene of FIG. 34A.

FIG. 34C shows an ultraviolet image (UV, 365 nm) of the scene of FIG.34A.

FIG. 34D shows a near infrared image (NIR, 980 nm) of the scene of FIG.34A, including a classification chart that can be used to detect liquidsin the scene of FIG. 34A.

FIG. 35A shows a visible spectrum image of a polystyrene foam platestreaked with sunscreen.

FIG. 35B shows a spectral image of the polystyrene foam plate of FIG.35A that is acquired using a filter element sensitive to an ultravioletwavelength at which sunscreen appears darkened.

FIG. 36A shows a visible spectrum image of a child wearing sunscreen.

FIG. 36B simulates the image of FIG. 36A captured at a wavelength atwhich the sunscreen appears darkened.

FIG. 37 simulates how the image of FIG. 36B may be highlighted to showprotected and unprotected skin areas.

FIG. 38A shows a visible spectrum image of a sunscreen surface hidden ina complex scene.

FIG. 38B shows how principles of the present invention may be used todetect and highlight the location of a sunscreen coated surface in thescene of FIG. 38A.

FIG. 38C is a visible spectrum, close up of the scene in FIG. 38A.

FIG. 39 suggests a means to automatically determine a light source fromits spectra for four common light sources.

FIG. 40 shows an illumination spectrum of sunlight and twocharacteristic wavelengths of the spectrum.

FIG. 41 shows a typical spectral sensitivity curve for a CCD sensoruseful in the practice of the present invention as well as twocharacteristic wavelengths of the spectrum.

DETAILED DESCRIPTION OF PRESENTLY PREFERRED EMBODIMENTS

The embodiments of the present invention described below are notintended to be exhaustive or to limit the invention to the precise formsdisclosed in the following detailed description. Rather a purpose of theembodiments chosen and described is so that the appreciation andunderstanding by others skilled in the art of the principles andpractices of the present invention can be facilitated.

An illustrative embodiment of a spectral imaging system 10 of thepresent invention is schematically shown in FIG. 1. Spectral imagingsystem 10 includes set-up features and operational features. The set-upfeatures, customized for each chosen target substance, involve obtainingthe electromagnetic spectrum for the chosen target substance, selectingunique spectral wavelength(s) to identify the substance, selectingcorresponding filter elements, and designing a unique target algorithmto reliably identify the target. The operational features include bothequipment and processing components. The operational equipment comprisesa spectral filter card 106 and a multi-camera array 111 having imagecapturing elements (such as CCD sensors). The operational processingcomponents may include the following processing steps: imageacquisition, image alignment, image normalization for variations inspectral parameters, computation of the presence and/or quantity of thetarget substance, generation of a colormap, and generation and displayof an output image showing the result of the detection procedure. Theoperational blocks are processing operations that may be executed at ornear real time as the multi-camera system is operating and the targetsubstance is within the field of view. While an advantage of the presentinvention is real-time operation, the present invention also encompassesmodes of operation in non-real time mode, batch mode, or other slow ordelayed processing operation. Additionally, the simultaneous imageacquisition permits freeze-motion images to be easily obtained formoving targets.

Illustrative set-up features of system 10 includes blocks (or steps) 21,22, 23, 24, and 25. The blocks 21, 22, 23, 24, and 25 are largely designsteps that help to establish the custom characteristics of the system 10for each desired application for which system 10 is used to detect oneor more target substances. FIG. 1 shows how system 10 is set up fordetection of a particular target substance. In practice, this designphase may be carried out as many times as desired to allow system 10 tobe used to detect a library of different target substances in multiplesettings.

Set-up begins in block 21 with a selection of a target substance whosedetection is desired. An advantage of the present invention is thatsystem 10 can be used to detect any target substance that has anidentifying electromagnetic spectrum in a reflection and/or transmissionmode when exposed to natural or artificial electromagnetic radiation.For example, some target substances exhibit characteristic spectralinformation at a plurality of different wavelengths when exposed tosunlight. Some substances exhibit characteristic spectral information atone or more particular wavelengths when irradiated with artificiallycreated infrared (IR), ultraviolet (UV), laser, or other irradiation.

Accordingly, it can be appreciated that the present invention can beused to detect a wide variety of different target substances.Illustrative examples include but are not limited to any of thefollowing target substances, either singly or in combination: water inany form (ice, steam, vapor, liquid, etc.), urine, arterial blood,venous blood, oxyhemoglobin, deoxyhemoglobin, fruits, vegetables, rodenttrails, pests, fingerprints, bodily fluids, gunshot residue, securityfeatures of a treasury bill or government check, sunscreen, ultravioletsecurity spray, ultraviolet paint, automotive fluids (i.e., oil,antifreeze, Freon), medical marker dyes, plants, minerals such as sand,quartz, alumina, other metal oxides, nitrides, carbides, ultravioletbrighteners that are present in hunters' clothing, ultraviolet dyes,tumors, skin cancer, wounds, chemical substances, solids, liquids(water, aqueous solutions, organic solvents, etc.), pastes, processedfoods, meats, body fluids containing a target substance that varies witha mammalian estrus cycle, grains, plant protein products, animal proteinproducts, animal complete feeds, mycotoxins, people, and animals.

In block 22, spectral information of the target substance is obtainedfor the target over desired range(s) of the electromagnetic spectrum. Asone example, an illustrative range of the electromagnetic spectrum canspan all or a portion of the wavelengths at least from about 200 nm toabout 14,000 nm. For reference, it is often convenient to divide thisspan of the electromagnetic spectrum into the following bands, UV, UVA,UVB, UBC, NUV, MUV, FUV, VIS, NIR, SWIR, MWIR, and LWIR.

The application of the present invention includes, but is not limitedto, all or portions of one or more of these bands. In many embodiments,the spectral information is obtained for at least a portion of thevisible band as well as at least a portion of at least one of theultraviolet band and/or the NIR band.

The spectral information for a target substance can be obtained usingany suitable spectroscopy technique(s) that provide spectral informationfor a plurality of wavelengths. In some embodiments, it is suitable toobtain a uniformly-sampled spectrum of the target over the desired rangeof the electromagnetic spectrum via hyperspectral imaging, multispectralimaging, or the like. Alternative systems to obtain a useful targetspectrum may include spectroscopy systems such as those commerciallyavailable from FOSS. Absorption and/or reflectance spectra may be usedfor the present invention.

In some modes of practice, spectral information of all or a portion ofthe anticipated background over a similar wavelength range andresolution is also obtained. This allows the spectral information of thetarget substance and its anticipated background to be compared. Fromthis comparison, spectral characteristics of the target substance,preferably at two or more wavelengths, can be identified that allow thespectral response of the target substance to be uniquely identifiedrelative to its background. For example, a particular crop of interestwhen healthy may have leaves that provide spectral peaks at 425 nm and705 nm wherein the magnitude ratio of the 425 nm peak to the 705 nm peakis on the order of about 2:1. However, when the crop is excessivelydehydrated, the ratio may drop to about 0.5:1. In such an instance, thehealthy plant may be the target substance (or vice versa), while thedehydrated plant is a background element (or vice versa). The ratio ofthe spectral responses of the crop at 425 nm and 705 nm can be used todetect healthy plants relative to dehydrated plants.

In block 23, the spectral information of the target substance andoptionally the background (if desired) is then analyzed via suitablesignal processing and spectral analysis techniques to uniquely identifyone or more, specific spectral wavelengths that may be used todistinguish the target substance from the background. In preferred modesof practice, at least two specific spectral wavelengths are identified.More preferably, 2 to 500, even more preferably 2 to 100, even morepreferably 2 to 50, and even more preferably 2 to 15 specificwavelengths are identified.

The concept of using spectral information of a target substance toidentify the wavelengths in block 23 is based on the fact that thespectra of common substances such as those shown in FIGS. 2A through 2Evary greatly when compared to the spectra of other substances. FIG. 2Ashows the absorption spectra of water. FIG. 2B shows that the spectraamong green grass, dry grass, and soil can be distinguished from eachother. FIG. 2C shows that the spectra of hemoglobin and oxyhemoglobinare distinguishable. FIG. 2E shows that spectra of tumors and normaltissue can be distinguished. In the practice of the present invention,these differences are used to facilitate detection. Conventionalhyperspectral imaging systems provide a full spectrum to distinguish onetarget substance from another. While effective, much of thehyperspectral information is superfluous.

In more preferred modes of practice, the present invention reduces theamount of spectral information used to make an identification to onlyselected portions of the spectrum that allow the desired detection to bepracticed. By reducing the acquisition and analysis data topre-selected, finite spectral features, great improvements in devicesize, cost, power consumption, resolution, and processing speed arerealized. Specifically, in preferred modes of practice the presentinvention reduces the spectral information needed to identify or measurethe target substance to that obtained for a discrete number of carefullyselected wavelengths or relatively narrow wavelength bands. The selectedspectral information is incorporated into the system design by assigningeach element of a spectral bandpass filter array to have a centerfrequency corresponding to one of the selected wavelengths or bands.This spectral filter array, implementing the selected wavelengthsensitivities, is combined with a multi-camera array such that eachspectral filter element is optically aligned with a correspondingelement of the multi-camera array. Thus, image acquisition from themulti-camera array results in an array of images with each representinga different wavelength sensitivity. These are useful to uniquelyidentify or quantify the desired target substance within the field ofview of the multi-camera array.

FIG. 3A illustrates a preferred approach used by the present inventionto identify pre-selected spectral wavelengths used to distinguish amongthe selected target substance(s) and/or background substance(s). FIG. 3Ashows how water can be distinguished from background spectralinformation using two spectral components. When the target spectrum andbackground spectrum are simple and substantially different as in FIG. 3Athe spectra need to be examined at only two wavelengths to easily detectthe target substance. In this example, absorption spectral informationcan be examined at 760 nm and 980 nm. If the ratio (980 nm/760 nm) orother suitable characteristic(s) of the spectral response at these twowavelengths is greater than a threshold, then the pixel of capturedimage information providing that response corresponds to water. If theratio is too low, no water is detected with respect to that pixel.

If the spectra are more complex and/or more similar, as in FIG. 3B, itis more desirable to select a greater number of wavelengths toaccurately detect the target substance. In this example, spectralinformation for durum wheat, triticale, barley, and individual durumkernels is shown. By obtaining spectral information at approximately 760nm, 1100 nm, 1200 nm, 1300 nm, 1460 nm and 1650 nm any one of thesubstances can be identified from the captured image information.Spectral properties of the captured wavelengths, e.g., ratios, sums,differences, averages, etc., can be analyzed to determine which of thematerials is associated with a pixel in the image information. Theassessment is accurate and fast. If computing power is fast enough, theresults of an analysis can be computed and displayed in a fraction of asecond even for images captured at higher resolutions, e.g., 16megapixels or higher resolution.

In both cases shown in FIGS. 3A and 3B the target substance can beaccurately detected with a relatively low number of spectral samples (2and 6 wavelengths, respectively, in these examples). The presentinvention may also be practiced by gathering spectral information at agreater number of wavelengths, but when the use of a more limited numberis effective, it is much more efficient and economical.

The advance selection of the filter wavelengths in the presentinvention, as part of the set-up, provides a significant advantage overconventional hyperspectral imaging systems in terms of the volume ofdata that must be acquired, stored and processed. A comparablemulti-camera system with 3 wavelengths requires less than 1% of the datavolume and processing effort when compared to a hyperspectral systemhaving 310 spectral bins. While maintaining the same diagnostic powerfor many substances, this data savings, inherent in the presentinvention, can be used to greatly increase the spatial resolution for amulti-camera system having the same memory space and processorcapability over its conventional hyperspectral counterpart.

Referring again to FIG. 1, in block 24, the selected wavelengths fromblock (23) are used to determine the corresponding filter elements (104)to be implemented as a filter array in the filter card (106). Generally,it is desirable to have at least one corresponding filter element 104for each selected wavelength identified in block 23 that will be used bysystem 10 to detect the target substance. For purposes of illustration,filter card 106 as shown includes an array of seven different filterelements 104. This indicates that for this particular target substance,spectral information at seven specific, different wavelengths are to beused to detect the target substance. For other target substances, thesame, a greater or lesser number of filter elements 104 may be includedin the array to accomplish detection.

Each filter element 104 is used to selectively filter the spectralinformation captured by the system 10 so that system 10 can assess whichportions (if any) of the captured image information generate spectralresponses at the selected wavelengths to confirm that the captured imageportion, e.g., each pixel of the image in some embodiments where pixelby pixel analysis is desired, indicates the presence of the targetsubstance at that location of the captured image. Desirably, each filterelement 104 has a bandwidth effective to capture one of thecorresponding, selected wavelengths to be used to detect the targetsubstance. More desirably, the center wavelength of each filter element104 is a selected wavelength determined in block 23.

The bandwidth of each filter element 104 may be selected to allow system10 to reliably detect and assess whether the specific spectral responseof the target substance at that wavelength is detected. Selecting anappropriate bandwidth for each filter associated with each selectedwavelength may be optimized as a tradeoff between a narrow bandwidth,which is more selective, and a wider bandwidth, which passes a greaterquantity of light making the system 10 perform better under low lightconditions. Balancing such concerns, a suitable bandwidth for eachfilter element 104 is typically up to 30 nm, or up to 20 nm, or up to 15nm, or up to 10 nm, or up to 5 nm, or even specific wavelengths in manymodes of practice. Often, a suitable bandwidth for each filter element104 is typically in the range from 10 nm to 30 nm.

For example, consider an exemplary spectral filter element with abandwidth of 10 nm wherein the bandwidth is centered at 500 nm. Such afilter element will selectively pass electromagnetic energy with awavelength of 495 nm to 505 nm while substantially blockingelectromagnetic energy having wavelengths above and below this range.Such a filter element is well suited for capturing image information atwavelengths of 495 to 505 nm so that system 10 can determine if anyportions of the captured image information indicate the spectralresponse of the target substance in this wavelength range. By using anarray of filter elements 104, system 10 selectively captures meaningfulspectral information at several wavelengths to enhance the ability ofsystem 10 to accurately detect the target substance.

As another specific example, if the selected application involvesdetecting the presence of water in a clear container via transmission oflight through the water within the container, the filter elements mightinclude the following:

-   -   a) a 980 nm bandpass filter with a 20 nm bandwidth, sensitive to        the water absorption band near 980 nm, making the water appear        very dark;    -   b) a 766 nm bandpass filter with a 20 nm bandwidth, sensitive to        a wavelength that does not include a water absorption band,        making it an excellent reference image that can see through        water; and    -   c) a visible spectrum bandpass filter which provides a spatial        orientation image such that the output of the processing of        images from a) and b) camera elements above may be oriented on        an image representing the field of view as seen by a human        observer.

Block 25 involves the generation of one or more algorithms that can beused to assess the captured, selectively filtered image information todetermine if the spectral information indicates the presence of thetarget substance from the anticipated background using the selectedwavelengths identified in block 23 and the corresponding filter elements104 that selectively capture image information at those wavelengths.Examples of algorithm methods include, but are not limited to,eigenvector, basis function, least squares, principle componentanalysis, ratios, differences, matched filter, neural networks,cross-correlation, multivariate analysis, and numerous classificationmethods.

The set-up of system 10 contemplated by blocks 21 to 25 can be carriedout for any number of additional target substances such that theresulting system 10 includes a plurality of spectral filter arrays andalgorithms to detect a substantial library of target substances. System10 can be configured to detect any target substance in the librarysimply by choosing the appropriate filter array and then using asuitable system interface to select the corresponding detectionalgorithm(s). The interface may be automated such that selection of aparticular filter array causes system 10 automatically to implement thecorresponding algorithmic analysis. Alternatively, in instances in whicha filter array might be useful for detecting more than one targetsubstance, the user could manually select the appropriate algorithm(s).In some embodiments, the orientation of a filter array when integratedinto system 10 may cause system 10 to automatically (or manually) selectalgorithm(s) corresponding to that orientation of the spectral filterarray.

It is anticipated that blocks 21 to 25 may be part of a manufacturingprocess for system 10 where an inventory of one or more spectral filtercards and corresponding algorithm(s) for a variety of differentdetection applications is designed and sold as a package. It is alsoanticipated that a family of filter elements may be incorporated into asingle filter card that contains subsets of filter elements, whereineach subset of the filter elements in the cards works to identify adifferent target substance.

For example, consider a filter card including four filter elements 1through 4, respectively. Application A may use filter elements 1, 2, and3 on a four-element filter card, while application B may use filterelements 1, 3 and 4, while application C may use filter elements 2, 3,and 4. The combination of filter elements for multiple applications isdependent upon the spectral characteristics of the selected targetsubstances and the desire of the user to have several applicationsreadily available without the need to change filter cards.

Another example of using a single filter card for multiple detectionpurposes might involve a three-application filter card designed forforensic investigators at a crime scene. Such a combination mightinclude the selection of spectral filter elements to detect bloodsplatter, filter elements to detect semen, and filter elements to detectfluorescent dyes used to highlight fingerprints. Quite easily, thepresent invention could provide three different filter cards (and theirassociated software algorithms) for blood, semen, and fingerprints,respectively, wherein each filter card includes an array of filterelements suitable for selectively capturing the previously identifiedset of selected wavelengths that allow the corresponding targetsubstance of interest to be identified from filtered image information.However, if the selected wavelengths to be analyzed overlap or aresubstantially the same (where uniqueness of each target substance isdetermined using an algorithm that analyzes characteristics of thespectral responses as a whole, such as ratios of responses at differentwavelengths, etc.), the present invention contemplates that a singlefilter array on a single filter card could be used to capture filteredimage information for all three substances. This is efficient if theplurality of target substances at issue have one or more filterwavelengths in common. In this case, additional functions can beprovided with only an incremental increase in the number of filter andcamera elements. Under the described combination scenario, differentfilter elements within the same filter card could be used by the same ordifferent algorithms to detect and locate each of the different targetsubstances in the captured image information. The output could comprisea color-coded image that displays the locations of blood, semen andfingerprints at the crime scene or any combination thereof.

A consumer example of a combination of applications on a single filtercard for a busy parent of small children with a kitten as a pet mightinclude a spectral filter group for detecting sunscreen (to assurecomplete coverage of children's skin at the beach), a filter group todetect cat urine stains (to quickly locate kitten ‘accidents’), and afilter group for matching paint colors (to precisely match paint colorsin the kitchen).

Block 26 indicates the optional step to calibrate the system forindividual variations in filter element and sensor element sensitivity.This step increases the accuracy of detection and is preferred. Thisfunction can be implemented as part of either the target algorithmdesign during manufacture of the system or later as part of the imagecapturing phase of using system 10. As an example of a desirablecalibration, each filter element 104 may have a specific attenuation oflight at its center frequency and across its bandwidth. This attenuationmay be measured during manufacture and recorded in a database for eachelement 104 of a filter card 106. The specific filter card calibrationcan be looked up in this database, over the internet or otherwise,according to a specific filter card serial number or designation. Acalibration, also stored in a database during manufacture, is possiblefor the sensor element(s) of the image capture array 111. Image capturearray 111 also is referred to herein as a “multi-camera array 111” toindicate that component 111 is capable of capturing a plurality offiltered spectral images simultaneously when the image capturingelements 112 are aligned with corresponding filter elements 104 in card106. Hence, each image capturing element 112 captures a corresponding,filtered image whose bandwidth is spectrally limited by the filterelement 104 through which the filtered image is captured. For purposesof illustration, each image capturing element 112 is associated with asingle camera body 114, and these bodies 114 are stacked in an arraysuitable to align with the corresponding filter elements 104. In othermodes of practice, multiple image capturing elements 112 may beincorporated into a common camera body. The term “multi-camera” stillapplies to a common body embodiment in the sense that “multi-camera”means that the system captures a plurality of images simultaneouslyregardless of the number of camera bodies involved. For example, FIG. 14shows an alternative embodiment of a multi-camera array including asingle camera body incorporating a plurality of image capturing elementsfor simultaneous image capture. As used herein, the term “multi-cameraarray” also is used to refer to an “image capture array”. Additionally,a “spectral filter array” also may be referred to as a “multi-elementspectral filter”.

The camera sensor elements may be calibrated using any suitabletechnique. For example, a multi-wavelength calibration is suitableeither via a calibrated light source that changes wavelengths, or astandard, broadband light source using a calibrated filter card withfilter elements at designated wavelengths and attenuations which permitan accurate interpolation across the entire spectral region for whichthe sensor is used. Both the filter elements and the camera sensorelements may also be recalibrated using the real-time processing systemand an automated calibration system used in the field. Such an automatedcalibration system desirably has the traits described above and canautomatically record the resulting calibration parameters within thememory of a smartphone, or other computing device to be used duringoperation of the system.

Once the spectral filter elements 104 in the filter card 106 have beenselected and provided, the target algorithm(s) developed and providedsuitable for detecting the corresponding target substance(s) at issue,and the system calibration optionally carried out, the system 10 canrapidly capture and process filtered images simultaneously acquired fromthe multi-camera array 111 to provide any desired output to report theresults of the detection. For example, the system can indicate whetheror not the target substance was detected. The system also may provideinformation showing the location of the target substance in the capturedimage information. For example, the system may provide a color-codedoutput image that shows where the target substance is within thecumulative fields of view of the camera array. Since the images from themulti-camera array may be sampled simultaneously, any motion of thetarget has less of an impact on detection accuracy as compared toinstances in which images or scans are performed only sequentially.Simultaneous image capture makes the present invention very suitable fordetection in environments with moving targets.

FIG. 1 shows how system 10 is used to capture image information from atarget surface 109 in Block 32 to allow system 10 to assess whether thetarget substance at issue is present in the target surface 109.Additionally, system 10 is used to capture the image information toallow system 10 to output the precise location(s) of the targetsubstance, if present, in the target surface 109. The images aresimultaneously acquired in step 32 from each image capturing element 112(also referred to as a camera element herein) of the multi-camera array112. Each image capturing element 112 is aligned with a correspondingfilter element 104 so that such image capturing element 112 capturesfiltered images through the corresponding filter element 104. Eachfiltering element 104 has a bandwidth that is sensitive to a uniqueselected wavelength associated with the detection of the targetsubstance. Therefore, each acquired image respectively includesinformation from only a unique, narrow region of the electromagneticspectrum centered and limited by the filter element 104 through whichthe image was captured.

Since the components of the multi-camera array occupy slightly differentspatial positions relative to the target surface 109, it may bedesirable in some modes of practice to more precisely align or registerthe captured images in optional block 33 such that corresponding pixelsin each image represent the same point on the target surface 109. Onceproperly aligned, the images may be further processed to detect thetarget substance on the target surface.

In preferred modes of practice, spatial image alignment or registration,is the process of overlaying or aligning two or more images of generallythe same scene taken from different spatial viewpoints. System 10, infact, simultaneously captures a plurality of filtered images ofsubstantially the same scene from a plurality of viewpoints. In order tomore accurately perform detection, it is desirable for the separateimages to be accurately aligned with each other such that correspondingportions of the target surface can be properly identified in each of thedifferent images. Advantageously, having at least two cameras capturingimages of the scene of interest from multiple viewpoints permits thepresent invention to acquire stereoscopic data. These data may be usedto derive three-dimensional data about the target surface usingstereoscopic image processing techniques known to those skilled in theart.

There are numerous methods available to accomplish this spatial imagealignment. A survey of suitable methods for spatial alignment orregistration of images is described in Barbara Zitova, Jan Flusser,Image Registration Methods: a Survey, Image and Vision Computing, 21(2003): 977-1000. The reviewed methods are area-based and/or featurebased. Registration is achieved generally according to four basic steps,namely, feature detection, feature matching, mapping function design,and image transformation and resampling. Different embodiments of thepresent invention may use one or more of the methods described in Zitovaand Flusser, as well as other registration methods known to thoseskilled in the art, to spatially align or register the multiple imagesacquired from the multi-camera array.

Since the spectral content that is reflected from or transmitted througha target surface may vary with the illumination spectrum, it may bedesirable in optional block 28 to properly identify the illuminationthat is present. This may be accomplished by examining the imageintensities from various spectral imaging elements 112 with knowledge oftheir associated filter wavelengths, or it may be accomplished byenergizing custom lighting with known spectral properties (i.e., LED,incandescent, sodium, neon, quartz halogen, ultraviolet, tungsten,fluorescent, compact fluorescent, infrared, near-infrared, mercuryvapor, xenon, or laser sources). Once the illumination spectrum isknown, the intensity of each acquired image may be normalized inoptional block 34 to take the spectral features of the givenillumination into account. A variation on the automated identificationof the illumination source would include a user interface (i.e.,software, switch or button) to manually identify the type ofillumination as shown by optional block 27.

In block 35, system 10 uses the captured image information, optionallymodified by processing such as alignment and normalization, to detectthe presence of the target substance (if any) on the target surface 109.System 10 then generates an output that communicates the results of theevaluation. A wide variety of different outputs can be used. Forexample, system 10 may output information indicative of whether thetarget substance was detected or not. Other output information may be inthe form of an image of the field of view of the system 10 that showsthe locations, if any, where the target substance is located. This isparticularly useful in applications where the target substance or itscharacteristics are not easily visible to the naked eye. Other outputmay communicate other qualitative or quantitative information concerningthe target substance.

Examples of color-coded output images are discussed below and shown inFIG. 4 (schematic), FIG. 5B, FIG. 17 (arteries and veins), FIG. 19B(organic contaminants on stainless steel), FIG. 20 (poultry carcass),FIG. 25E (water), FIG. 27 (water), FIG. 28C (grains), FIG. 32B (roadconditions), FIG. 33 (car vision), FIGS. 34B to 34 D (liquids), FIG. 35B(sunscreen), FIG. 36B (sunscreen), and FIG. 37 (sunscreen).

In another type of “presence” output, a displayed image may beformulated to show the spatial presence of the target substance as asolid color easily discerned from other parts of the display. Thedisplayed image may be processed to indicate the presence of the targetsubstance above a threshold level as a binary image mask. This mask isthen used to force the associated pixels in the spatial orientationimage to a fixed level. The colormap may be programmed to provide agrayscale color for all levels except the fixed level corresponding tothe pixels related to the target substance.

In another type of “quantity” output, a displayed image may beformulated to show a quantitative amount of the target substance atlocations within the target field of view. A binary target mask mayfirst be derived as described for the ‘presence’ above. However, afurther processing step introduces varying values in the ‘presence’locations that represents the concentration or quantity of the targetsubstance. The colormap is then derived to transform the variations inquantity into variations in color or color intensity. In some modes ofpractice, data used to generate the custom ‘presence’ and ‘quantity’displays may be made available for analysis or export to anothercomputing device.

FIG. 1 shows an exemplary technique for generating a “presence” outputin the form of a displayed image in which locations of the targetsubstance in the displayed image are colored substantially differentlyfrom other portions of the image. According to this technique, theacquired images can be converted in block 37 into an output image 122 ofthe target surface 109 showing the locations 120 of the target substanceon the image 122. The matrix of pixels making up image 122 may beenhanced to more clearly show the locations of the target substance bycolor-coding the image. If this is desired, a suitable color map may beselected in optional block 36 and then applied in block 37 to output thecolor-coded image 122. As a result, the locations 120 of the targetsubstance are shown as areas of a different color relative to the otherportions of image 122. In many embodiments, the color used to representthe locations of the target substance can be much brighter than theother colors in image 122 to make it easy to see the locations 120 inimage 122.

The image information can be evaluated at any desired resolution. Forexample, each pixel of the captured image information can beindividually analyzed. This is a highly accurate way to identify notonly that the target substance was detected but also to preciselyindicate where the location(s) of the target substance is in thecaptured image information. In other modes of practice, the system 10might analyze a plurality of pixels. This is an extremely accurateapproach to detect the presence of the target substance in the image,but the location of the target substance is less precise than if eachpixel is evaluated.

The present invention can acquire images and display results atextremely fast rates. For example, in some modes of practice, system 10executes the necessary computations and processing steps to provide adisplayed image as output within a time period from image capture thatis less than the frame rate required for video imaging, typically 1/30second. This allows the present invention to operate and display resultsat a video rate or higher. This is accomplished according to oneillustrative approach by applying the algorithms of this invention tothe simultaneously acquired, multi-camera images via a high-speed, imageor signal processor. Since the images of this invention are all acquiredsimultaneously, this invention can accurately freeze motion within thefield of view preserving relative positions of moving targets within thefield of view, a feat impossible for competitive systems thatsequentially apply filter elements in front of a single camera. Thiscapability of the present invention provides an accurate, real-timerecord of a target that is moving.

A variation on the above embodiment can provide a video playback rateusing a slower processing system. Since the images are acquired andsaved simultaneously, a slower processing system can be used to applythe algorithms of this invention in slower than real time. While theslower processing system does not process the data fast enough to keepup with the actual event, the fully-processed, final video record can beplayed back at a video (or higher) rate to accurately represent theoriginal motion of the target.

System 10 also is useful in any non-contact, application where aninstantaneous chemical analysis of a remote surface is beneficial. Insome embodiments, system 10 can be mounted on dynamic platforms thatmove in the environment. Examples include land, air, and water-basedvehicles. For example, system 10 can be mounted on piloted or unmannedaircraft to capture aerial images of a region of interest. The capturedimages can be used on the aircraft and/or can be transmitted to a remotelocation, if desired.

A key feature of system 10 is a multi-camera array capable of capturinga plurality of filtered images simultaneously. System 10 cansimultaneously capture all spectral components needed for detectionsince all the spectral imaging elements 111 are optically aligned withcorresponding filter elements 104 at the same time. Images may becaptured simultaneously, providing advantages for high-speed imaging andimaging of moving targets.

The present invention uses terminology to characterize the images usedby system 10. A detection image (DI) is an image acquired by a spectralimaging element 112 and its corresponding filter element 104. Eachcorresponding spectral imaging element 112 and its corresponding filterelement 104 constitute a detection pair. Each detection pair has aselected wavelength sensitivity that is strategically influenced by thepresence of the target substance. The filter element 104 of such a pairis selected to provide a wavelength sensitivity for detection of aunique wavelength at which the target substance has a known spectralresponse as determined in block 23. In order for accurate information tobe determined for the target substance, system 10 desirably includes atleast one detection pair to capture at least one detection imagerepresenting a strategic wavelength useful to detect the targetsubstance at issue.

A spectral reference image (SRI) is an image acquired by a detectionpair with a selected wavelength sensitivity that is minimally influencedby the presence of the target substance. The spectral reference image isuseful for evaluating illumination and establishing reference levelsessential to detecting changes due to the target substance. As describedfurther below with respect to water, such a wavelength also may be usedto identify a target substance based upon the minimal spectral responseat that wavelength.

A spatial orientation image (SOI) is an image that reasonably representsthe spatial orientation of objects within the cumulative fields of viewof the system 10 as viewed within the visible spectrum. The SOI may beacquired by an independent element of the system 10 or it may be ashared image also serving as a spectral reference or detection image.

An output image (OI) 122 is an image that provides the output of theoperational processing and that shows the derived information about thetarget substance. In one embodiment, the output image includes a spatialorientation image, shown generally in grayscale except that the targetsubstance information is displayed as a color-coded addition to thegrayscale SOI in those places in the image where the target substancehas been detected.

To summarize, system 10 may be adapted to a specific application via aset-up or design step that selects specific wavelengths that uniquelydistinguish the spectrum of a target substance from the spectra ofbackground substances. System 10 formulates a target algorithm thatidentifies the target substance from its spectral characteristics at aplurality of selected wavelengths. A multi-camera array is thenimplemented having elements that are sensitive to each of the selectedwavelengths, respectively. The unique spectral sensitivity for eachelement of the multi-camera array is achieved via an interchangeablefilter array with each filter element optically aligned with acorresponding element of the multi-camera array. Each element of themulti-filter array passes only one of the selected wavelengths. Thisstructure converts the universal multi-camera array into a customspectral analysis tool specifically designed to analyze the spectrum ofthe chosen target substance. The derived information regarding thetarget substance(s) has many useful applications.

FIG. 4 schematically shows an alternative embodiment of a system 200according to the present invention. System 200 includes camera body 202incorporating a plurality of spectral imaging elements 204 that capture,encode, and store filtered images captured by system 200. In manyembodiments, a spectral imaging element 204 includes a lens or acombination of lenses that is optically coupled to media capable ofchemically and/or electronically storing images transmitted by the lensor lenses to the media. Exemplary electronic media include sensors suchas a charge-coupled device (CCD) or CMOS image sensor. CCD and CMOSsensors are commonly used in digital photography with a blocking filterto exclude wavelengths outside of the visible spectrum. A widerbandwidth with these economical sensors may be achieved in the presentinvention by removing or not installing these blocking filters. Othersensors also may be used. These include those based on indium galliumarsenic (InGaAs), indium antimonide (InSb), mercury cadmium telluride(HgCdTe), lead selenide (PbSe), microbolometers, doped silicon and amultitude of other sensor compositions to provide a desired spectralsensitivity in a selected portion of the electromagnetic spectrum. Thespectral imaging elements 204 may capture images using any suitableimaging methodology such as a single shot capture system. For purposesof illustration, system 200 includes three spectral imaging elements204.

A filter array 206 comprising filter elements 208 is incorporated intosystem 200. The filter array 206 includes a filter element 208 for eachspectral imaging element 204. For purposes of illustration, one of thefilter elements 208 has a spectral bandwidth of 420 nm to 430 nm that iswell suited for detecting a spectral response at 425 nm, λ₁. A second ofthe filter elements 208 has a bandwidth of 570 nm to 580 nm that is wellsuited for detecting a spectral response at 575 nm, λ₂. The third filterelement 208 has an optical bandwidth of 630 nm to 640 nm that is wellsuited for detecting a spectral response at 635 nm, λ₃.

System 200 is being used to analyze whether a target substance havingpre-determined spectral characteristics at 425 nm, 575 nm, and 630 nmmight be present in a surface 212 and, if present, where the targetsubstance is located. System 200 is aimed so that the spectral imagingelements 204 have a cumulative field of view 211 that encompassessurface 212 with multi-wavelength light 207 reflecting from the surfaceinto spectral filter 206. For purposes of schematic illustration, system200 has a resolution such that surface 212 constitutes a 3×3 array ofpixels in the captured images. For convenience, the pixels are labeledas pixels a through i, respectively. In actual practice, captured imageswill include thousands and even millions of pixels.

System 200 is actuated to capture filtered images 214, 216, and 218.Captured image 214 is captured to assess the spectral response of eachpixel at 425 nm, λ1. Captured image 216 is captured to assess thespectral response of each pixel at 575 nm, λ2. Captured image 218 iscaptured to assess the spectral response of each pixel at 635 nm, λ3.Each of the captured images 214, 216, and 218 is processed to align the3×3 array of pixels in the captured images. For illustration purposes,each pixel of each captured image having a pre-defined spectral responseassociated with each targeted wavelength is marked. According to thealgorithm used for this detection, the target substance is deemed to belocated in a particular pixel if that pixel in all the images is marked.System 200 then provides an output image 220 showing that the targetsubstance was detected in the surface 212 at locations corresponding topixels b, d, and e.

Another embodiment of a spectral imaging system 230 of the presentinvention is shown in FIGS. 5A, 5B, and 5C. System 230 includes a camerabody 232 wirelessly linked to smartphone 233. Camera body 232 includes aplurality of image capturing elements 234 in a recess 236. Recess 236 isbounded by frame 238 that can be used to hold any one of interchangeablefilter cards 240 from filter card library 242. Each filter card 240includes a base 243 and a plurality of filter elements 244. As anoption, some filter cards may have light sources 245, such as LED,installed for one or more wavelengths. For purposes of illustration, onesuch filter card 240 is being inserted into frame 238. Upon insertion,each of the filter elements 244 on the card 240 optically aligns with acorresponding image capturing element 234. Upon insertion into thecamera body 232, system 230 automatically identifies the filter card andselects an algorithm suitable for analyzing captured images to detectone or more target substances associated with that filter card 240. Allof the filter cards 240 in library 242 are interchangeably installedinto camera body 232 so that system 230 is easily configured to detect awide variety of target substances corresponding to the filter cards 240and corresponding algorithms.

Smartphone 233 provides computing power, display for input and output,and an easy touch screen interface for system operations. In oneembodiment, the present invention is physically attached to smartphone233 or other battery-powered mobile device such that the batterycapacity within the present invention may be shared with the smartphone233 or mobile device by connecting the power circuits of the devices ortransferring energy via magnetic field link or other suitable coupling.

System 230 can capture a plurality of filtered images (not shown) from ascene 246 and 293. In the embodiment of FIG. 5B, the scene 293 includestarget substances 295 and non-target substances 294. System 230 detectsthe differences in the spectral signatures of the substances 294 and 295within the scene 293, and displays an output image showing the locationof the target substances 295 in a different color 296 than thenon-target substances 297 on the display 298 of the smartphone 233. Theregions of the display that do not contain the target substance may bedisplayed in grayscale 297.

System 230 allows specific wavelengths in a portion of theelectromagnetic spectrum to be selected to uniquely identify the targetsubstance at issue. Other illustrative applications of this inventionmay include medical imaging, crop analysis, sanitation, egg inspection,food processing, meat processing, vascular imaging, wound healing, skincancer and tumor detection, urine analysis, moisture detection, buildinginspection, vegetation analysis, forensic detection of body fluids,manufacturing, pharmaceutical, nutraceutical, machine vision, mineralscreening, sunscreen detection, estrus detection, pest control, hunting,and carpet stain detection.

The ability to capture and evaluate selected wavelengths for differenttarget substances is incorporated into the system via filter elements244 of the spectral filter array included in card 240 and a customtarget algorithm that evaluates captured image information to assess thespectral characteristics corresponding to those of the target substance.With this invention a new target substance may be detected by simplyusing a different, interchangeable spectral filter card 240 and adifferent target algorithm associated with the new card. Preferredfeatures of system 230 include the use of a multi-camera array capableof simultaneous image acquisition, an interchangeable filter card,economical charge-coupled device (CCD) detectors (commonly used indigital cameras), calibration methods, high resolution, and displayswith custom colormaps. These features help the present invention bringthe analysis power of a remote-sensing satellite or hyperspectralimaging system to an economical, compact platform.

The interchangeable filter cards 240 are an advantageous feature ofsystem 230. Choosing among the different interchangeable filter cards240 can quickly change the spectral characteristics of system 230 todetect different target substances. Individual filter elements 244 ofthe cards 240 are optically aligned with corresponding spectral imagingelements 234 to allow the acquisition of images that are sensitive toselected wavelengths useful for the accurate detection of a targetsubstance. System 230 is easily configured to detect a different targetsubstance simply by changing the filter card 240. Optionally,interchanging a new filter card 240 may cause system 230 toautomatically select an associated target algorithm that has beenspecifically designed for the newly installed card and its correspondingtarget substance(s). For example, the present invention may include anindication on the filter card, such as a model number or radio-frequencyidentification (RFID) tag or the like, that can be read electronicallyand used by the software of the present invention to automatically loadthe proper target algorithm and user interface from electronic memorystorage resident within the device or located in memory accessible via anetwork connection, internet, cloud storage, or the like. Thus, as a newfilter card is slid into place, the device senses the change, loads theproper target algorithm and modified user interface, such that thesystem is ready to detect the new target substance with maximumconvenience for the user.

Embodiments of the present invention such as system 230 areadvantageously derived from CCD-based camera systems, as these arewidely available at low cost. These cameras often contain a visiblespectrum filter that cuts out ultraviolet light and near infrared light,leaving the camera sensitive to the visible spectrum, approximately400-700 nm. By removing (or not installing) this filter, the nativesensitivity of the CCD sensor is expanded to approximately 300-1100 nm,i.e., from ultraviolet, through visible, to near infrared. Thus,incorporating a plurality of such CCD-based camera systems into amulti-camera array provides a low-cost, high-resolution imaging systemsuitable for analyzing spectral information at a plurality of selectedwavelengths from a relatively wide wavelength range of approximately 300to 1100 nm. This wavelength range is very suitable for detecting a widerange of target substances. As an alternative to CCD-based sensors, orin addition to CCD-based sensors, other embodiments of the presentinvention may incorporate near infrared sensors that are sensitive towavelengths longer than 1100 nm. In another aspect, system 230 canaccommodate multiple selected wavelengths by using different kinds ofsensors sensitive to different bands. For example, one spectral imagingelement sensitive to 600 nm may be implemented via a CMOS sensor. Asecond spectral imaging element sensitive to 800 nm may be implementedvia a CCD sensor. A third array spectral imaging element sensitive to1500 nm may be implemented via a GaAs-based sensor.

Advantageously, system 230 incorporates and converts a common smartphoneinto a high-resolution, chemical imaging tool. Additionally, the smallsize and low-power requirements make the device ideally suited formobile, aerial or stationary operation. The universal platform, using amulti-camera array with interchangeable filter elements, permits theeconomies of high-volume manufacturing to be achieved for a base unitwhile still enabling a wide range of target substances to be detected bysimply changing filter arrays. This invention may also be used withother computing devices such as tablet, mini-tablet, laptop and desktopcomputers. System 230 may be connected to any number of outside devicesvia connectivity standards common to smartphones and computing devices.Examples of such connectivity interfaces include, but are not limitedto, Wi-Fi, Bluetooth, and/or numerous additional wireless standards. Inanother embodiment, the wireless link may be replaced by any suitablewired link. Examples of wired links are selected from at least one ofUSB, Firewire, Ethernet, custom, proprietary, or a multitude of otherwired standards.

Available light sources vary in their spectral output. Natural sunlightprovides a wide range of spectral output sufficient for mostapplications. However, artificial lighting might include incandescentlighting, fluorescent lighting, white LED lighting, compact fluorescentlighting or other light sources. Each of these artificial light sourcesprovides a narrower output spectrum than natural sunlight. For examplean incandescent light provides good illumination in the visible and nearinfrared, but provides a lower intensity in the ultraviolet spectrum.Fluorescent light provide visible and some ultraviolet illumination, butexhibits lower output in the near infrared region. Consequently, in someembodiments, system 230 may be used in combination with artificial lightsources to help minimize the impact of different lighting situationsupon detection accuracy. For example, the interchangeable filter cards240 optionally may contain LED or other illumination sources to moreeffectively and uniformly illuminate the field of view at the desiredwavelengths. Power to the illumination source(s) may be obtained fromthe system 230 power supply, a separate power source, the basesmartphone device, or even a battery supply incorporated into anothercomponent of system 230. Other auxiliary light sources may be selectedfrom any number of light sources, such as incandescent, ultraviolet,near infrared, infrared, tungsten, mercury vapor, sunlight, Xenon,quartz halogen, compact fluorescent, high pressure sodium, and metalhalide, or combinations of these. The auxiliary light source(s), if any,may be positioned in an advantageous position in order to illuminate thetarget field of view. The auxiliary light source(s), if any, may also becontrolled by the present invention.

The advent of inexpensive, high-resolution, miniature cameras for thesmartphone and security markets provides powerful components to couplewith the image processing strategy of the present invention. Miniaturecameras with 8 to 13 megapixel resolution are now common. Rather thansequentially swapping filters in front of a single large, expensivecamera, it is now cost-effective to operate an array of miniature,high-resolution image capturing elements in parallel. Using such amulti-camera array, the present invention includes an interchangeablespectral filter array on a module or card that can be placed in front ofthe multi-camera array. When the characteristics of each filter elementare selected as previously described, it becomes possible toconveniently change the spectral characteristics of the multi-camera,multi-filter arrays. A new target algorithm to optimize the informationobtained from the target substance can be readily installed either bymanual or automated selection such as from an electronic directoryresident within the device or located in memory accessible via a networkconnection, internet, cloud storage, or the like. The automatedselection of the proper target algorithm may be triggered by anidentification characteristic, such as a part number, on the newlyinstalled filter card or module.

With this interchangeable filter and algorithm capability, the abilityto detect additional target substances can be implemented quickly byexamining the spectral characteristics of the new target substance,determining the necessary wavelengths and filter requirements, designingthe new target algorithm, then using a new filter card and applying anew software algorithm to optimally extract information from the newtarget substance.

In some modes of the present invention, an alternative embodiment ofsystem 230 may be available having a relatively low number of imagecapturing elements 234 and spectral filter elements 244, for example 2to 4 of such element pairs 234/244. FIG. 5A schematically illustratessuch an embodiment having three spectral imaging elements 234. Suchembodiments are capable of detecting information from a modest set oftarget substances. In other embodiments of the present invention,another embodiment of system 230 may be available having a higher numberof spectral imaging elements 234, for example 5 to 12 spectral imagingelements. Such higher element embodiments will be capable of detectinginformation from a larger set of target substances.

In some embodiments, system 230 optionally includes a global positioningsystem (GPS) capability to geographically locate the images acquired. Toaccomplish this, system 230 may use the GPS capability of the linkedsmartphone 233 or other suitable device. System 230 also may use thedirectional orientation capabilities of the linked smartphone 233 orother device to determine the elevation and direction that the cameraarray is aimed. In a specific application of these principles, system230 may use GPS capabilities (or even manual entry) to locate cropvegetation data and orient these data on a map of the crop field. Cropfeatures may be color coded or otherwise marked to indicate the healthof the crop at various locations in the field. An accumulation ofmultiple images may be used to cover large portions of a crop or even anentire field.

As an option, system 230 may incorporate electronically, tunable filterelements as all or part of one or more spectral filter arrays. Suchtunable filter elements could replace all or part of the interchangeablefilter cards 240 with an array of tunable filters for which the desiredwavelengths may be electronically selected. Optionally, any detectionpair (i.e., a particular filter element 244 and its corresponding imagecapturing element 234) may further incorporate at least one polarizingor other kind of spectral filter within the optical path, either on theinterchangeable filter card or as part of the multi-camera array, or asa separate component, to reduce glare or otherwise modulate spectralinformation from the target field of view. Any spectral variationintroduced by the polarizing filter desirably is accounted for via asuitable calibration process.

In FIGS. 5A and 5B, camera body 232 is shown as a separate hardwarecomponent that is mounted to smartphone 233. In other embodiments,multi-camera capability may be incorporated directly into a smartphone,mobile device, tablet, mini-tablet, laptop computers, desktop computeror other computing device to provide a spectral imaging system of thepresent invention in a single, integrated device.

In another embodiment, the camera body 232 and smartphone 233 need notbe physically coupled together. Instead, these may be operated at adistance from each other. Camera body 232 and smartphone 233 maycommunicate with each other in any suitable way such as by a wirelessand/or wired connection. In such a configuration portions of the userinterface and processing may be conveniently distributed in either partor allocated between both parts of the system.

The integration of smartphone 233 into system 230 provides moreadvantages. Optionally, a color or grayscale camera integrated into thesmartphone 233 may be used as part of the multi-camera array to providea spatial orientation image when displaying detection results. Forexample, pixels of the spatial orientation image corresponding to thedetected target substance can be highlighted via color or other indiciato show the precise locations of the target substance in the spatialorientation image. Additionally, the camera system within the smartphonemay be used as a viewfinder to aim the smartphone/multi-camera assembly,whereby the multi-camera system is used to acquire at least a portion ofthe captured images when the system is actuated.

Spectral imaging systems of the present invention generally include twoor more detection pairs of camera elements and corresponding spectralfilter elements. FIGS. 6, 7, 8 and 9 shows alternative embodiments ofspectral imaging systems 260, 262, 264, and 266 of the present inventioncontaining different numbers of detection pairs. System 260 includesmulti-camera array 268 and filter array 270. System 260 is an example ofan embodiment in which multi-camera array 268 includes two cameraelements. Filter array 270 includes two filter elements, each aligningwith a corresponding camera element. System 262 includes multi-cameraarray 272 and filter array 274. System 262 includes three detectionpairs of camera elements and corresponding filter elements. System 264includes multi-camera array 276 and filter array 278. System 264includes seven detection pairs of camera elements and correspondingfilter elements. System 266 includes multi-camera array 280 and filterarray 282. System 266 includes 100 detection pairs of camera elementsand corresponding filter elements.

FIG. 10 shows another embodiment of a spectral imaging system 300 of thepresent invention in which spectral imaging arrays 302 and 304 aredeployed at a plurality of different locations relative to a scene 314being imaged. Spectral imaging array 302 includes multi-camera array 306and filter array 308. Arrays 306 and 308 provide seven detection pairsfor purposes of illustration. Spectral imaging array 304 includesmulti-camera array 310 and filter array 312. Arrays 310 and 312 provideseven detection pairs for purposes of illustration. In this embodiment,the two spectral imaging arrays 302 and 304 are positioned and aimed toexamine two sides of a target (not shown) in scene 314.

An advantage of the present invention is that many different kinds ofcamera elements may be used in the spectral imaging systems. FIG. 11shows a typical embodiment of a camera element 320 including lensassembly 322 and image sensor 324. Typically, a plurality of cameraelements 320 are appropriately coupled to a common circuit board 326.Circuit board 326 may serve not only a plurality of camera elements butalso other components of a spectral imaging system. In otherembodiments, each camera element 320 may incorporate its own dedicatedcircuit board. Lens assembly 322 and image sensor 324 may be mounted tocircuit board 326 as shown or may be remote and coupled to board 326 bysuitable wired or wireless interconnects.

FIG. 12 shows an example of a multi-camera array 330 including 6detection pairs. Array 330 includes six lens elements 332 and acorresponding array of six image sensors 334. The lens elements 332 andsensors 334 are mounted on a common circuit board 336.

FIG. 13 shows an example of a spectral imaging system 350 whosedetection capabilities are easily configured to detect a wide range oftarget substances. System 350 includes multi-camera array 351 andspectral filter array 358. Multi-camera array 351 includes a lens arrayof lens elements 352 and corresponding sensor array of sensors 354. Lenselements 352 and sensors 354 are coupled to circuit board 356. Filterarray 358 includes card 362 incorporating filter elements 360. Eachfilter element 360 aligns with a corresponding lens element 352. Filterarray 358 allows the multi-camera array 351 to be sensitive to capturingimage information for a plurality of pre-determined wavelengths. Thewavelengths are pre-selected so that the captured image information canbe used to detect a target substance in the field of view of the system.Filter array 358 is easily interchanged with another filter array (notshown) of similar size and similar distribution of filter elements sothat the other filter array aligns with the lens array. By using adifferent combination of filter elements in the other filter array, thespectral sensitivity and hence detection capabilities of the system 350is expanded to encompass one or more other target substances. Byproviding a larger inventory of such arrays, and by providingprogramming instructions that allow the system to interpret the spectralinformation captured when using the various filter arrays, system 350can be used to detect a vast number of target substances on demand.

FIG. 14 shows an example of a spectral imaging system 380 includingspectral imaging capabilities based on 120 detection pairs. System 380includes spectral filter array 382, lens array 384, sensor array 386,and circuit board 396. Filter array 382 includes a 12×10 array ofspectral filters 388 supported on substrate 390. Filter array 382 isinterchangeable with other filter arrays (not shown) to expand thedetection capabilities of system 380. Lens array 384 includes a 12×10array of lens elements 394 that are optically aligned with correspondingfilter elements 388. Sensor array 386 includes a 12×10 array of imagesensors 398. The image sensors 398 are aligned with corresponding lenselements 394.

The focusing of the images onto the image capture elements within theimage capture array of the present invention may be accomplished inaccordance with any suitable method(s). Examples include, but are notlimited to, manual focus, numerous autofocus methods and focus methodsunder the control of program instructions. The focus of an individualdetection pair, also referred to as a channel, may be made independentlyor in coordination with one or more other channels.

An advantageous characteristic of many embodiments of the presentinvention, including but not limited to those embodiments as illustratedin FIGS. 12 and 13, is that chromatic aberration is reduced or evenavoided. Chromatic aberration is a common distortion of optical systemsusing lenses. This distortion plagues many conventional spectral imagingsystems, such as those that use a spinning wheel of filter elementspassing different wavelengths through a single lens and single cameralens system. The refractive index of lens materials varies withwavelength. This causes different wavelengths to be focused by a singlelens at different distances from the lens. Camera manufacturers havedesigned various complex and expensive lens configurations to minimizethis distortion with only marginal success over a broad range ofwavelengths. The present invention solves this problem, even for simple,inexpensive lenses, since the present invention has a separate spectralchannel for each selected wavelength. The optimal focus for eachwavelength channel may be established and left in place while acquiringimages. This permits the present invention to acquire imagessimultaneously and at video or faster frame rates. There is no need tosequentially refocus the system between each new wavelength acquisition,as is the case with conventional technologies.

In some modes of practice, the present invention provides a solution tochromatic aberration by providing independent filter/camera channels(i.e., detection pairs) for each selected wavelength, permitting thedistance from a lens to the focal plane to be adjusted independently toproperly focus the spectral image on the image plane of the imagecapture sensor.

In some modes of practice, an optional correcting lens may be includedas part of the interchangeable spectral filter card, which corrects fordeviation in focus due to the wavelength of the specific spectral filterelement.

In some modes of practice, an optional complex lens design thatcompensates for variations in wavelength and refraction index may beused as part of the present invention.

In some modes of practice, individual lenses in an image capture arraymay be adapted, e.g., offset relative to a common image plane or have analtered curvature, etc., so that each lens in the array can capture andindependently focus filtered light onto the image plane. Such anadaptation is desirable because each lens is intended to capturefiltered light in a particular bandwidth that may require a differentfocal length than other bandwidth portions captured by other lenses inthe array.

In some modes of practice, an aperture of a lens/camera system isadjusted to increase the depth of field and reduce the chromaticaberration with minimal change in focus.

In some modes of practice, the present invention may determine a focusfor one channel and then compute a corresponding focus for additionalchannels based on the corresponding lens characteristics and the centerwavelength of each corresponding spectral channel.

One of the strengths of the present invention is its flexibility ineasily generating target algorithms for a wide variety of targetsubstances. The form and design of a target algorithm has fewrestrictions. The target algorithm may be formulated to take advantageof the characteristics of the target spectrum and the number ofwavelengths available in a given product model. Some target substancesmay be easily detected or measured using only a few selectedwavelengths, e.g., 3 wavelengths. More complex detection efforts, e.g.,wherein a target substance has many spectral similarities to otherfeatures in the captured image information, may involve using a greaternumber of selected wavelengths, e.g., 10 or more, for detection.

For example, when the principles of the present invention are used todetect water as described herein and depending upon other materials inthe background, a simple ratio between the spectra sampled at twodifferent wavelengths would be quite effective to detect water in afield of view of a spectral imaging system. If spectral information fora particular image pixel demonstrates such a ratio, the system mayaccurately conclude that water is present in the captured image at thatpixel location. If the ratio is not demonstrated, there is no waterdetected at that pixel location. The system can perform this analysisfor every pixel (or at another resolution, if desired) tocomprehensively detect and map the location(s) of water throughout thecaptured image information. Even with this simple example, there isgreat flexibility in the selection of the wavelengths and the manner inwhich they are compared to identify or measure the target substance.

For an exemplary goal, such as to identify a given target substanceusing two wavelengths, one algorithm criterion may involve theidentification of a narrow spectral peak in the spectrum of the targetsubstance. This criterion is based on the principles that backgroundsubstances are likely to have different peaks or broader transitions. Tothis end, two parameters are selected to optimize, namely the ratiobetween the spectra amplitudes and the separation of the wavelengths,with a narrow separation being preferred. It is possible to optimizeboth parameters by writing the exemplary formula below which defines athree-dimensional, optimization surface:

$\begin{matrix}\begin{matrix}{{S\left( {\lambda_{i} - \lambda_{j}} \right)} = {\left\lbrack {{R\left( \lambda_{i} \right)}/{R\left( \lambda_{j} \right)}} \right\rbrack/{\left( {\lambda_{i} - \lambda_{j}} \right)}}} & {\lambda_{i} = {{\min\left( \lambda_{1} \right)}\text{:}{\max\left( \lambda_{1} \right)}\mspace{14mu}{and}}} \\\; & {\lambda_{j} = {{\min\left( \lambda_{2} \right)}\text{:}{\max\left( \lambda_{2} \right)}}}\end{matrix} & (1)\end{matrix}$where λ₁ and λ₂ are the range of the first and second wavelength,respectively; λ_(i) and λ_(j) are the indexed wavelengths within therange of λ₁ and λ₂, respectively; R(λ_(i))/R(λ_(j)) is the ratio of thereflectance spectrum at indexed wavelengths; and S(λ_(i)−λ_(j)) is theoptimization surface. The optimization surface, S(λ_(i)−λ_(j)), may thenbe searched for a maximum. The wavelengths near that maximum arecandidates to become the selected wavelengths for implementation in themulti-filter array and the target algorithm. A similar strategy may beused for more than two wavelengths. Numerous other formulas and methodsmay be useful to select a given number of wavelengths which may be used,in turn, to identify the chosen target substance.

Once the wavelengths are selected as suggested above, signal processingand statistical analysis methods may be used within a target algorithmto detect the target substance using spectral information at theselected wavelengths. Such methods include, but are not limited to,eigenvalue analysis, correlation analysis, principle component analysis,signal detection theory, pattern recognition, and multivariate analysis.An important feature of the present invention is the great freedom itprovides to use a variety of effective target algorithms to accuratelydetect the target substance while still providing a platform that iseconomical, low-power, mobile, and high resolution.

In some modes of practice, the target substance may be a pure substance.In other modes of practice, the target substance may be a mixture of oneor more substances. The design of the target algorithm(s) can easilyaccommodate mixtures, because mixtures often exhibit spectralinformation that is different in a plurality of respects from the puresubstances that make up the mixture. In the case of such a mixture,spectra may be acquired at various concentrations and analyzed to createcorresponding target algorithm(s) capable of detecting and evenproviding quantitative information for mixture variations. This strategymay be used to advantage to actually measure the concentration of atarget substance by evaluating the spectra at the selected wavelengthsfor each level of concentration. One application of this technique is toremotely and noninvasively measure moisture content in any number ofsubstances such as soil, sand, grain, cereal, vegetation, feed, or wallmaterial. An analysis of the spectra in the manner described formixtures can also be used to evaluate the purity of a target substanceor determine what impurities may be present.

The present invention, therefore, may be used to detect not only thepresence of a target substance in a field of view but also to measure aquantity of the target substance within the field of view when spectralcharacteristics vary based on quantity. While a presence may beaccurately detected using an accumulation of nominal values for filterattenuation, sensor variations, and light source variations, it may beadvantageous to use an in-frame reference surface to calibrate theseparameters across the formulation range of interest when makingquantitative measurements. The in-frame reference may be used as part ofa calibration procedure to quantify the concentration of the targetsubstance. The spectral imaging system can use that information tocalibrate the images captured with the in-frame reference in the fieldof view. For example, a certain ratio between peaks at two or morewavelengths may indicate a concentration of 10.0%, whereas another ratiomight indicate a concentration of 25.0%, etc. This allows quantitativeinformation to be determined with great accuracy.

System calibration may be achieved via a number of methods. One suchmethod is a cumulative calibration factor computed from individualcalibration factors determined for each system component such asillumination intensity, filter attenuation, lens attenuation, and imagecapture sensor sensitivity. Using this method, the overall systemcalibration is the cumulative product of the individual componentcalibration factors. Another method of calibration involves the use ofan in-frame reference with known reflective properties on its surface.The in-frame reference method permits the entire system to be calibratedas a whole and avoids a possible accumulation of errors which may occurwith the cumulative calibration factor method. The optimal calibrationmethod for a given application depends on the required accuracy for thatspecific application and the cost or complexity of the calibrationmethod.

In the cumulative calibration method, one component calibration is forillumination. An exemplary method to obtain a calibration for variationsin illumination provides for the image data at each selected wavelengthto be normalized to account for variations in illumination near thatparticular wavelength. In practical effect, this may be described as amultiplication by an illumination normalization factor, k_(illum). Thispermits an accurate comparison of the spectrum of the target substanceat the selected wavelength without undue influence from variations inthe illuminating source across the spectrum.

As an example, if the illumination from sunlight is as shown in FIG. 40,an illumination normalization factor, k_(illum), may be computed at eachselected wavelength to remove the spectral response introduced by theillumination that is independent from the spectrum of the targetsubstance.

In this example, the relative illumination at two wavelengths, 550 nmand 980 nm, respectively, are used to demonstrate the suggestednormalization. These wavelengths are shown in FIG. 40.

$\begin{matrix}{I_{550} = {1.00\mspace{14mu}\left( {{at}\mspace{14mu} 550\mspace{14mu}{nm}} \right)}} & (1) \\{I_{980} = {0.586\mspace{14mu}\left( {{at}\mspace{14mu} 980\mspace{14mu}{nm}} \right)}} & (2) \\{{k_{illum}\left( {550\mspace{14mu}{nm}} \right)} = {\frac{1.0}{I_{550}} = 1.000}} & (3) \\{{k_{illum}\left( {980\mspace{14mu}{nm}} \right)} = {\frac{1.0}{I_{980}} = 1.706}} & (4)\end{matrix}$

In general terms, the illumination normalization factor, k_(illum), as afunction of wavelength, λ, may be expressed as the reciprocal of theillumination curve, I(λ), at each wavelength.

$\begin{matrix}\begin{matrix}{{k_{illum}\left( \lambda_{i} \right)} = \frac{1.0}{I\left( \lambda_{i} \right)}} & {{for}\mspace{14mu}{non}\text{-}{zero}\mspace{14mu}{I\left( \lambda_{i} \right)}}\end{matrix} & (5)\end{matrix}$

Illumination curves may be selected from a database list of genericillumination sources such as sunlight, incandescent light, fluorescentlight or more specific light sources such as “a 60-watt, soft white GElight bulb” or “tungsten lamp, model 6315 1000 W QTH Lamp3.” Theseillumination curves may reside in a memory base such as, local memory ofthe present invention, local memory of the associated mobile device, oran on-line or “cloud” database to which the device has access via theinternet or other means. In any of these cases, the illumination sourcedata will be such that the relative illumination intensity may bedetermined at the specific wavelength of a given camera/filter element.

In the cumulative calibration method, another component calibration isfor sensor sensitivity. An exemplary method to obtain a calibration forvariations in sensor sensitivity provides for the image data near eachselected wavelength to be normalized according to the sensor spectralresponse via multiplication by a sensor normalization factor,k_(sensor)(λ_(i)), where λ_(i) is the ith wavelength used to define thetarget substance (with other parameters at known levels or alreadynormalized). This permits an accurate comparison of the spectrum of thetarget substance at each of the selected wavelengths.

As an example, for the spectral sensitivity of a CCD sensor as shown inFIG. 41, a sensor normalization factor, k_(sensor)(λ_(i)), may becomputed at each selected wavelength to remove the sensor's spectralresponse from the evaluation of the target substance spectrum.

In this example, the sensitivity at two wavelengths, 550 nm and 980 nm,respectively, are used to demonstrate the suggested normalization. Thesewavelengths are shown in FIG. 41.

$\begin{matrix}{S_{550} = {0.620\mspace{14mu}\left( {{at}\mspace{14mu} 550\mspace{14mu}{nm}} \right)}} & (6) \\{S_{980} = {0.818\mspace{14mu}\left( {{at}\mspace{14mu} 980\mspace{14mu}{nm}} \right)}} & (7) \\{{k_{sensor}\left( {550\mspace{14mu}{nm}} \right)} = {\frac{1.0}{S_{550}} = 1.613}} & (8) \\{{k_{sensor}\left( {980\mspace{14mu}{nm}} \right)} = {\frac{1.0}{S_{980}} = 1.222}} & (9)\end{matrix}$In general terms, the sensor normalization factor, k_(sensor)(λ), as afunction of wavelength, λ, may be expressed as the reciprocal of thesensor sensitivity curve, S(λ), at each wavelength.

$\begin{matrix}\begin{matrix}{{k_{sensor}\left( \lambda_{i} \right)} = \frac{1.0}{S\left( \lambda_{i} \right)}} & {{for}\mspace{14mu}{non}\text{-}{zero}\mspace{14mu}{S\left( \lambda_{i} \right)}}\end{matrix} & (10)\end{matrix}$

The sensor response curve may be determined for each individual sensordevice as part of the manufacturing process where that sensor curve isstored in a memory base, such as a device memory or may also be acquiredby looking up an identification number for the device in an on-line or“cloud” database to which the device has access via the internet orother communication system. In any of these cases, the sensorsensitivity data will be such that the relative sensor sensitivity maybe determined at the specific wavelength of a given camera/filterelement.

Additionally, the sensor sensitivity curve may be re-measured andrecorded in a memory base such as, local memory, an on-line database, ora ‘cloud’ database and made available for calibration purposes. Thisrecalibration of the device may be performed at regular intervalsdetermined by the aging characteristics of the sensor.

In the cumulative calibration method, another component calibration isfor filter gain. Each discrete narrow band spectral filter used with acamera element typically has some losses. To account for these losses, afilter normalization may be practiced that uses the filter gain,A_(filter), for each filter element. As an example, the discretewavelengths of 550 and 980 nm are used.

$\begin{matrix}{A_{550} = {0.85\mspace{14mu}\left( {{at}\mspace{14mu} 550\mspace{14mu}{nm}} \right)}} & (11) \\{A_{980} = {0.80\mspace{14mu}\left( {{at}\mspace{14mu} 980\mspace{14mu}{nm}} \right)}} & (12) \\{{k_{filter}\left( {550\mspace{14mu}{nm}} \right)} = {\frac{1.0}{A_{550}} = 1.176}} & (13) \\{{k_{filter}\left( {980\mspace{14mu}{nm}} \right)} = {\frac{1.0}{A_{980}} = 1.25}} & (14)\end{matrix}$

In some modes of practice, the system calibration may be accomplishedvia an accumulation of normalization factors specified for eachcomponent in the system. This method is represented by the formula belowfor each spectral component.I _(cal)(λ_(i))=k _(illum)(λ_(i))k _(sensor)(λ_(i))k _(filter)(λ_(i))I_(measured)(λ_(i))  (16)where the normalization factors, k_(illum), k_(sensor), and k_(filter)are as described above; I_(measured) (λ_(i)) is the uncalibratedintensity as measured at wavelength, λ_(i); and I_(cal) (λ_(i)) is thecalibrated intensity for the target surface at the designatedwavelength, λ_(i).

In general terms, the filter normalization factor, k_(filter)(λ), (as afunction of wavelength, λ, may be expressed as the reciprocal of thefilter gain, A_(filter)(λ), at each wavelength

$\begin{matrix}\begin{matrix}{{k_{filter}\left( \lambda_{i} \right)} = \frac{1.0}{A\left( \lambda_{i} \right)}} & {{for}\mspace{14mu}{non}\text{-}{zero}\mspace{14mu}{A\left( \lambda_{i} \right)}}\end{matrix} & (15)\end{matrix}$

The cumulative calibration method may be advantageous for the detectionof the presence of a target substance. The normalization factors for theillumination source, sensor sensitivity, and filter element gain may beretrieved from calibration data stored in memory. The illuminationsource may be selected from a list of typical sources which may beencountered such as sunlight, incandescent, fluorescent, tungsten, LED,or other light sources with commonly known spectral emissions. Thesensor calibration may be obtained by an initial calibration at the timeof manufacture, periodically updated as needed. The filter efficiency isavailable as a specification for each individual filter element, perhapsconfirmed at the time of manufacture.

In illustrative modes of practice, an in-frame reference may be used tomore precisely calibrate all system components at once. Such acalibration may use a single surface (or multiple surfaces), broadband,in-frame reference (or reflector) with known spectral characteristics.In the case where the in-frame reference spectrum is uniform at theselected wavelengths, the system calibration vector, k_(system)(i), maybe computed according to the following formula:

$\begin{matrix}{{k_{system}(i)} = {{\frac{1}{I_{ref\_ surf}(i)}\mspace{20mu}{for}\mspace{14mu} i} = {1\mspace{14mu}{to}\mspace{14mu} n}}} & (18)\end{matrix}$where i is an index representing each multi-camera capturing element(and its corresponding filter element); n is the number of elements inthe multi-camera array (excluding the spatial orientation image elementif it presents the entire visible spectrum); and I_(ref_surf)(i) is theintensity of the in-frame reference surface as acquired in the image ofeach element of the multi-camera array.

As an example, for a four-element, multi-camera array including fourdifferent wavelengths, a calibration array might be written, usingequation 18, as:

$\begin{matrix}\begin{matrix}{K_{system} = \left\lbrack {k_{system}(1)} \right.} & {k_{system}(2)} & {k_{system}(3)} & \left. {k_{system}(4)} \right\rbrack \\{= \left\lbrack 1.732 \right.} & {1.523} & {1.122} & \left. 2.455 \right\rbrack\end{matrix} & (19)\end{matrix}$where the system calibration constants, k_(system)(1) throughk_(system)(4), are the calibration constants for each image acquiredfrom each multi-camera element, respectively. These calibrationconstants represent a calibration for the entire spectral path of thegiven camera/filter element pair. In this example, the calibrationimages, Image(i)_(cal), obtained from each uncalibrated (raw) image,Image(i)_(raw), may be computed in the following manner:Image1_(cal) =k _(system)(1)Image1_(raw)=1.732×Image1_(raw)Image2_(cal) =k _(system)(2)Image1_(raw)=1.532×Image1_(raw)Image3_(cal) =k _(system)(3)Image1_(raw)=1.122×Image1_(raw)Image4_(cal) =k _(system)(4)Image1_(raw)=2.455×Image1_(raw)  (20)with the corresponding selected wavelengths expressed by a wavelengthvector:Λ=[λ₁λ₂λ₃λ₄]=[550 650 830 980] nm  (21)In cases where the spectrum of the in-frame reference is not uniformacross the selected wavelengths, the measured spectral amplitudes may benormalized according to variations in reference levels and then theformulas above may be applied.

The in-frame reference calibration method has the potential for greatersystem precision since undue accumulation of normalization factor errorsis avoided by accounting for the normalization factors all at once. Thismethod is particularly advantageous for applications where aquantitative measurement of the target substance is desired.

It should be noted that the two described embodiments for systemcalibration proposed above should yield nearly equivalent results withthe second, the in-frame reference method, expected to be somewhat moreprecise. Therefore,k _(system)(i)˜k _(illum)(λ_(i))k _(sensor)(λ_(i))k_(filter)(λ_(i))  (22)where k_(system)(i) is the system normalization for the ith cameraelement sensitive about wavelength, λ_(i); with the remainder of thenormalization factors as previously described.

It may also be desirable to adjust the acquired images for individualcamera optics such as aperture, shutter speed, depth of field, andfocus. One solution is to link the optical and spectral features of thecamera array such that they all change in unison. Another solution is tofix the camera optics with the same parameters. A third solution is topermit each element to optimize certain parameters which aremathematically accounted for within the calibration or measurementalgorithms. Throughout the normalization and calibration process it isdesirable to account for variations in intensity such that the relativespectral amplitudes at various wavelengths are maintained.

Advantageously, the present invention may be used in reflectance and/ortransmission modes. In reflectance mode the light reflects from thetarget surface and is detected by the system components. In thetransmission mode, the light is transmitted through the target surfaceand then continues on to be detected by the sensors within the system.Subjects such as vegetation, minerals, fresh produce, and fluorescentpaints typically are viewed in the reflectance mode. Subjects such asliquids or gasses are often viewed in a transmission mode. Somesubstance may be viewed in either mode individually or in both modessimultaneously, such as mammalian skin and tissue. It is important touse the proper spectrum for reflectance or transmission (absorption)modes, respectively, because transmission and reflectance spectra for aparticular substance may not be the same. For example, spectral peaks ina transmission spectrum may be at different wavelengths and/or havedifferent amplitudes than spectral peaks in a reflectance spectrum.

One embodiment of the present invention uses a permanent, stationarygeometry of the multi-camera array. In this embodiment, all elements ofthe multi-camera array are spatially fixed with respect to each other(such as by permanent mounting each camera element on the same circuitboard). Thus, a registration between images from various camera elementscan be determined at the time of manufacture.

In another embodiment, a reference target surface with an advantageoustarget subject matter may be used to derive a registration formula tofully register the images from various camera elements. With thisembodiment, the registration or alignment of the various images from theindividual camera elements can be empirically determined at the time ofmanufacture and an alignment formula stored in digital memory. Such analignment formula can then be recalled as needed throughout the lifetimeof the product.

An advantageous alignment target, used to optimally register or alignthe individual images, may be selected or created to have similarfeatures visible to all camera elements regardless of their spectralsensitivity. Using this alignment target, images can be acquired thatare ideal for generating an alignment mapping function. An alignmentmapping function can be generated for each camera element within thearray, transforming them to a common or reference axis.

Another embodiment of the present invention may use focus or zoomcapability. In this embodiment a stationary mapping function is usedwith an added mathematical formula to compensate for a change in focusor zoom. Such an additional formula will include an image scalingcomponent to account for a change in image size due to focus or zoomcapability.

Advantageously, the present invention provides a common platform capableof performing spectral image analysis for numerous, diverse detectionand analysis applications. Exemplary applications include, but are notlimited to, chemical composition analysis, water and moisture detection,water analysis, crop analysis, vegetation identification, vegetationmapping and infestation tracking, soil analysis, disturbed soilanalysis, cranberry analysis, inspection of fruits and vegetables,locating missing persons, inspection of nuts, seeds, legumes and grains,robotic harvesting of fresh produce, biofilm detection, hand washing,food safety and processing, infection control, egg inspection, birdvision, forensics and criminology (e.g., detection of fingerprints,urine, blood, semen, hair, and the like), vascular imaging, diabeticfoot perfusion, peripheral vascular disease, wound healing, feedback forcardiopulmonary resuscitation (CPR), cardiovascular risk assessment viaa reactive hyperemia protocol, surgical tool inspection, surgical toolto indicate vascular clamping, lymphatic imaging and surgery, evaluatingperipheral artery disease (PAD), breast and other cancer detection,medical imaging, medical diagnostic information automatically linked toan individual subject, veterinary and human applications, estrusdetection, building inspection, floor cleaning, pest control, vehiclediagnostics, detection of fluorescent dyes, sanitation, mineraldetection and processing, frac sand supply and use, meat processing,poultry processing, sunscreen analysis, animal feed inspection andanalysis, security and forgery detection, authentication, hunting,consumer color matching, tattoos and other body art, gas detection,liquid classification, oil production, chemical processing,manufacturing, pharmaceuticals, nutraceuticals, sustainability, roadconditions, and the like.

Chemical Analysis

In one aspect, the present invention relates to a method to acquiresimultaneous images in order to analyze the chemical composition of atarget surface within a field of view.

Water and Moisture Content

In one aspect, the present invention relates to a method to deriveinformation about water within a target field of view. If the targetsubstance is a volume of water, the selected wavelengths might be onewavelength in which water is absorbed, such as 980 nm, and onewavelength in which water exhibits little absorption, such as 766 nm.Filter elements would contain bandpass filters for each of the selectedwavelengths. After image alignment, a simple ratio of image intensitiesfor corresponding pixels would yield an indication of the presence ofwater.

In another aspect, moisture may be detected by comparing imageintensities at two or more different wavelengths where the wavelengthsare selected by an analysis of the spectral reflectance properties ofmoisture. After image alignment, the presence of moisture may bedetermined by the ratio of image intensities at the differentwavelengths. Ratios greater than a given threshold could be indicativeof the presence of moisture.

In another aspect, the ratios described above for the detection ofmoisture may be calibrated via an in-frame reference surface in order tocalibrate a given ratio value with a given percentage of moisture. Sucha calibration may be used to determine a quantity of moisture presentwithin the target surface. Such applications are valuable for measuringthe moisture content of many substances, such as soil, cereal, foodproducts, meat, snow, crops, vegetation, sheet rock, concrete, chemicalproducts, frac sand, building construction, and minerals.

Water Analysis

In another aspect, the present invention relates to a method to analyzesubstances present in water. As discussed herein, the spectralabsorption band of water near 980 nm is narrow. This characteristicpermits the identification of water with a filter element sensitive near980 nm. Adjacent bands, outside of the water absorption band, areavailable to detect impurities, contaminants or formulations within thewater. Examples of such uses include but are not limited to, pollutiondetection, on-site water analysis, aerial inspection of water quality,beverage processing, wastewater treatment, drinking water treatment,biosecurity at reservoirs, toxicity analyses, presence of organicmaterial, presence of organisms, groundwater analysis and underwaterexploration or mining.

Crop Analysis

In one aspect, the present invention relates to a method to analyze acrop. In such an application the multi-camera array is aimed at a cropin order to derive information regarding the condition of the crop. Thismay include, but is not limited to, a crop condition such as nitrogencontent, chlorophyll content, crop maturity, moisture content, diseasestate, insect infestation, fungus infestation, mold content, mildewcontent, weed content, plant maturity, harvest readiness, and fertilizereffectiveness.

FIG. 15 illustrates the use of a spectral imaging system 400 of thepresent invention to analyze a crop 402 within a field of view 404.System 400 includes an image capture array 406 including a trio of imagecapture elements 408. System 400 also includes a filter card 410including spectral filter array 412 that includes a trio of filterelements 414. The filter elements 414 are selected to be sensitive tothe desired crop characteristics. When filter card 410 is inserted intosystem 400, filter elements 414 optically align with image captureelements 408 to allow each of the image capture elements 408 to capturean independent, filtered image of crop 402. System 400 may furtherinclude a smartphone (not shown) integrated with image capture array 406and spectral filter array 412. The spectral imaging system 400 ispositioned at an elevation sufficient to view at least a portion of thesurface of crop 402 within a field of view 404. A corresponding targetalgorithm is provided to analyze the acquired spectral images andcompute the desired crop information.

Crop data acquired via system 400 in FIG. 15 may be communicated to anynumber of outside devices via connectivity standards common tosmartphones and computing devices. Examples of such connectivityinterfaces include, but are not limited to, Wi-Fi, Bluetooth, and/ornumerous additional wireless standards. If useful, the wireless link maybe replaced by any suitable wired link. Examples of wired links areselected from at least one of USB, Firewire, Ethernet, custom,proprietary, or a multitude of other wired standards.

Positioning and aiming the system 400 at the crop 402 may beaccomplished in a variety of ways, including one or more of securing theinvention within or on an aerial apparatus such as a commercialaircraft, private aircraft, glider, satellite, spacecraft, unmannedaerial vehicle, remote control aircraft, drone, blimp, lighter than airaircraft, manned balloon, weather balloon, projectile, rocket, personalair vehicle, paraglider, kite, or extraterrestrial aircraft; attachingthe invention to a flying animal such as a bird or bat; placing theinvention in the possession of a skydiver deployed over the crop;attachment to a pole, building, greenhouse, hill, mountain, tree, crane,bridge, overpass, or other permanent, semi-permanent, or temporarystructure; attachment to a water tower, cell phone tower, or electricaltower; photographic tripod, mechanical fixture or similar apparatus;handheld or attached to a human; attached to a person via a mechanism,such as a head strap or chest strap; attached to or held by an animal;attached to or held by a robot; attached to a ground-based equipmentsuch as a car, tractor, combine, harvester, plow, irrigation apparatus,spraying system or remote control vehicle; and positioning the inventionin any manner that, at least temporarily, enables the crop to be withinthe field of view of the present invention.

In addition to being used for crop analysis, such positions are alsosuitable for other applications where an elevated position may beadvantageous to capture images of a scene for detection of a desiredtarget substance.

In another aspect, the present invention contains geographicalcoordinates for each set of images representing a portion of the fieldof view such that the images and data derived from the images may beassembled into a contiguous collage representing a larger portion of theentire crop field than is contained in any single image set. Thesegeographical coordinates may be embedded in the image data, as ispresently done via geo-tagging protocols, or organized in an independentdata base.

Many crops may benefit from the use of the present invention. Thisinvention may be applied to any crop that exhibits a spectral responsethat can be characterized by the spectral amplitudes at a discretenumber of wavelengths. Examples of such crops include corn, soybean,wheat, rice, cotton, cranberries, grapes, rye, sorghum, canola, rapeseed, peas, sugar beets, oats, alfalfa, sugar cane, tomatoes, potatoes,edible beans, coffee, oranges, grapefruits, apples, nuts, peanuts,legumes, strawberries, blueberries, blackberries, onions, tobacco,peppers, spinach, broccoli, carrots, grass, brome, lupines, and avocado.

Crop analysis may include, but is not limited to, soil analysis, soilanalysis for the purpose of fertilizer distribution, growth tracking,plant maturity analysis, insect infestation, disease, nutrient content,stress levels, mold detection, mildew detection, weed analysis, cropdusting, harvest readiness, and the like. In another aspect the presentinvention may be used to detect the signature of a specific crop for thepurpose of analyzing the percentages of crop types in a given region.

In another aspect, the present invention may be used to detect thesignature of a specific illegal crop, such as marijuana, opium poppy(heroine) or coca (cocaine). Such use of the present invention maybenefit law enforcement when used for aerial inspection of a givenregion or for illicit crop documentation in a handheld device.Additionally, farmers may inspect their fields for unauthorized illegalcrops planted in their fields, a practice commonly used by ‘guerrilla’growers of illegal crops. The present invention is particularly wellsuited for this application in that it has the analysis power ofconventional hyperspectral imaging at a great reduction in cost, and asignificant increase in resolution over competitive hyperspectralimaging systems. The increase in resolution permits law enforcement toconduct surveillance from higher altitudes. A higher surveillancealtitude reduces the risk from retaliatory shootings by illegal growers.A higher altitude, above 500 feet, also minimizes court challenges forviolation of privacy, sometimes encountered by law enforcement when theyconduct low-altitude surveillance.

FIG. 16 shows examples of spectra for healthy vegetation, unhealthyvegetation and soil. Three wavelengths (A, B, and C) are shown whichenable the discrimination between the conditions. In the practice of thepresent invention, detecting information at these three wavelengths isenabled by using three camera and filter elements with a filtersensitivity centered around each of the designated wavelengths. In theexample of FIG. 16, it is possible to determine if a given pixelrepresents soil or vegetation by computing the ratio of wavelength C towavelength B, C/B. Those pixels with a C/B ratio above threshold (i.e.,2.0) represent vegetation while those pixel with a lower C/B valuerepresent soil pixels. Only those pixels representing vegetation need beanalyzed for vegetation health.

Vegetation Identification

In another aspect, the present invention relates to a method ofidentifying specific species of vegetation within the field of view.This application goes beyond the crop identification described above.Various species of vegetation have different spectral responses, makingit is possible to identify many plant species via an analysis of theirspectral response. Examples include, but are not limited to, treespecies identification in a forest, crop species identification in anagricultural region, and grassland species documentation.

In another aspect, the present invention relates to a method to identifygenetically modified organisms (GMO) where the genetic modificationcauses a change in the target spectrum as related to the unmodifiedorganism (i.e., GMO corn or wheat detection).

Vegetation Mapping and Infestation Tracking

In another aspect, the present invention relates to a method to map aspecific species of vegetation. For example, the present invention mayacquire aerial data indicating the location of various tree specieswithin a forested, rural, urban or suburban setting.

In another aspect, the present invention relates to a method to map aspecific species of vegetation for the purpose of pest control. Thepresent invention may acquire aerial data indicating the location ofvarious tree species within a forested, rural, urban or suburban settingwhich are susceptible to infestation. For example, the location of ashtrees may be determined which are susceptible to infestation by theEmerald Ash Borer beetle.

In another aspect, the present invention relates to a method to trackpest infestation within a specific species of vegetation. For example,the present invention may acquire aerial data indicating the location ofash trees which are susceptible to the Emerald Ash Borer beetle andfurther analyze these data to identify damage to these same ash treesindicative of actual infestation. This is possible because the spectralcharacteristics of damaged trees would be distinct from that of healthytrees.

Soil Analysis

In another aspect, the present invention relates to a method to analyzesoil. The spectra of various soils may be evaluated to determine theselected wavelengths that distinguish between the soil types ofinterest. Filter elements are then chosen that are sensitive to theseselected wavelengths.

In another aspect, the present invention relates to a method to analyzethe soil in preparation for crop planting and selection of fertilizernutrients and dosage.

Disturbed Soil Detection

In another aspect, the present invention relates to a method to detectdisturbed soil. The soil analysis capability described herein may beapplied to a region of soil within the field of view. The target soilarea is analyzed. Any digging in the soil is likely to leave traces, orlarge amounts of soil that was originally positioned at a depth belowthe surface. This variation in surface soil composition can be detectedwith the present invention and highlighted on a display of the soilanalysis. Unnatural soil discontinuities may automatically be flagged aspossible disturbed soil.

Applications of disturbed soil detection may include, but are notlimited to: the detection of dangerous devices, at or below the soilsurface, which are intended to harm nearby subjects; the detection ofunderground tunnels; the location of shallow graves or buried objects;the location of archeological artifacts and structures from pastcivilizations; and the early detection of illegal drug cultivation.Disturbed soil detection may also be useful in the management practicesof mining, agriculture, and construction.

Cranberry Industry

Cranberries are a valuable crop for which the present invention hasparticular importance. While the total acreage of cranberries planted inthe U.S. is a small fraction of other crops such as corn, soybeans orwheat, the per acre value of cranberries is much greater. The 2013cranberry revenue was reported to be $9566 per acre in comparison to$960, $536 and $468 per acre for corn, soybeans and wheat, respectively.(See USDA Noncitrus Fruits and Nuts, 2012 Preliminary Summary, Jan. 1,2013). Due to stringent wetland laws, cranberry growers find itdifficult to expand their acreage to meet increasing demand for theirproduct. Therefore, it is advantageous for cranberry growers to increasecurrent crop yield in existing cranberry beds.

In one aspect, the present invention relates to a method to mapcranberry fields for soil analysis, maturity tracking, diseasedetection, weed management, and harvest readiness. The high resolutionof the present invention makes the present invention attractive wherethe lower resolution of conventional satellite and other conventional,remote sensing technologies fall short.

In another aspect, the present invention relates to a method tointegrate a telephoto lenses into one or more spectral imaging elementof a camera array. With this additional feature, a crop field can bescanned with even greater resolution. The use of telephoto lenses on thecamera elements can be beneficial in many applications other than cropscanning where high resolution scanning from a distance is desired.

In another aspect, the present invention relates to a method to screenthe harvested berries. Cranberries typically reach receiving stationswith various grades of berry and numerous contaminants mixed with theproduct. Most cranberries are harvested by flooding the fields,vibrating the berries from the vines, and corralling the berries withfloating booms towards the waiting harvest machines along the edge ofthe flooded field. The harvested cranberries may be white (unripe) andvarious shades of red. Rotten or bruised berries are also present. Sincecranberry beds are sometimes adjacent to roadways, highway litter mayend up at the berry processing plant in the form or paper, plastic,cigarette butts, and even metal particles embedded in the berries. Thesevarious berry conditions and contaminants each have different spectralsignatures that can be used to differentiate them from the desired goodberry. Used in conjunction with automated sorting machines, the presentinvention can provide a low-cost, high-resolution solution to thesorting needs of the cranberry industry.

In another aspect, the present invention relates to a method to providea pre-screening system positioned adjacent to the cranberry fieldsduring harvest that is low-cost, portable and of high-resolution. Thispre-screening system would screen and remove defective berries andcontaminants before loading on trucks for transport.

Inspection of Fruits and Vegetables

In another aspect, the present invention relates to a method to inspectproduce, such as fruits and vegetables, for contaminants within aquantity of the produce or on the surface of the produce. Spectra of thedesired fruit or vegetable are acquired and compared with acquiredspectra of likely contaminants. Selected wavelengths of these spectraare analyzed as described herein to determine the presence ofcontaminants within a quantity of the produce or on the surface of theproduce. This information may be provided in real-time to automatedsorting equipment to remove the contaminant from the good produce.

In another aspect, the present invention relates to a method to inspectproduce, such as fruits and vegetables, for quality, such as ripeness,maturity, bruising, rotten spots, mold, mildew or other imperfections.Spectra of the ideal produce are analyzed along with spectra of eachimperfection. Selected wavelengths of these spectra are analyzed asdescribed herein to determine the presence of produce imperfections.This information may be provided in real-time to automated sortingequipment to remove the imperfect item. In a similar manner variousgrades of the produce may be identified and sorted accordingly.

Inspection of Nuts, Seeds, Legumes and Grains

In another aspect, the present invention relates to a method to inspectnuts, seeds, legumes and grains for contaminants within a quantity ofthe product or on the surface of the product. Spectra of the desirednut, seed, legume or grain are acquired and compared with acquiredspectra of likely contaminants. Suitable wavelengths of these spectraare analyzed as described herein to determine the presence ofcontaminants within a quantity of the product or on the surface of theproduct. This information may be provided in real-time to automatedsorting equipment to remove the contaminant from the good product.

In another aspect, the present invention relates to a method to inspectnuts, seeds, legumes and grains, such as peanuts, walnuts, cashews,pecans, almonds, hazelnuts, chestnuts, acorns, pistachios, Brazil nuts,beechnuts, sunflower seeds, rice, pine nuts, ginkgo nuts, bunya nuts,macadamia nuts, garden or crop seeds, grass seed, wheat, rye, corn,soybean, barley, sorghum, canola, rape seed, oats, beans, and coffee,for quality, such as ripeness, maturity, bruising, rotten spots, mold,mildew or other imperfections. Spectra of the ideal produce are analyzedalong with spectra of each imperfection. Selected wavelengths of thesespectra are analyzed as described herein to determine the presence ofproduce imperfections. This information may be provided in real-time toautomated sorting equipment to remove the imperfect item. In a similarmanner various grades of the produce may be identified and sortedaccordingly.

Robotic Harvesting of Fresh Produce

Robots have been developed to harvest delicate fresh produce crops.However, present machine vision technologies do not have the ability toeffectively distinguish between ripe and unripe fruits and vegetables,nor can they reliably identify leaves, stems and branches. The presentinvention fills a serious need that exists for an economical, mobile,imaging system to provide the ability to reliably select ripe fruits andvegetables, as well as exclude leaves, stems and branches.

In another aspect, therefore, the present invention relates to a methodto provide a machine vision tool that can distinguish between ripe andunripe fruits and vegetables. It can also differentiate between produce,leaves, stems and branches. By analyzing the spectra of each item therobot is likely to encounter, it becomes possible to determine thespatial position of each item within the robot's field of view. Thefilter card can be designed to provide each camera element with theappropriate wavelengths to identify the desired produce components. Whencompared to a similar, conventional hyperspectral imaging system, thelow cost and high resolution of the present invention are advantageousfor this application. The much lower data volumes of many embodiments ofthe present invention also facilitates easier real-time processing. Inanother aspect, the present invention relates to a method to provide theabove features as a system that is not mobile, but stationary.

Biofilm Detection

A biofilm is any group of microorganisms on a surface. These adherentcells are frequently embedded within a self-produced matrix ofextracellular polymeric substance (EPS). Biofilms may form on living andnon-living surfaces and can be prevalent in many environments, such as,natural, industrial, agricultural and hospital settings. Biofilm EPS,often referred to in the common vernacular as slime, often is aconglomeration of extracellular DNA, proteins, and polysaccharides.

While the microorganisms that generate a given biofilm require amicroscopic analysis to directly detect and identify the species, thereis evidence that the chemical composition of the biofilm generated by agiven species of microorganism may be unique and can be readily viewedby macroscopic spectral analysis. In a recent study two genera ofmicrobial biofilms have been identified on a stainless steel surfacewhich is commonly used in food processing systems via conventionalhyperspectral imaging methods. Won Jun, Moon S. Kim, Kangjin Lee,Patricia Milner, and Kuanglin Chao, Assessment of Bacterial Biofilm onStainless Steel by Hyperspectral Fluorescent Imaging, Sens. & Instrum.Food Qual., 2009, 3:41-48, DOI 10.1007/s11694-009-9069-1,http://afrsweb.usda.gov/SP2UserFiles/person/35964/2009JunEtAI3-1SensInstrumFoodQualSafety41-48.pdf,viewed Nov. 3, 2013. In this study hyperspectral imaging was used tosuccessfully detect biofilms generated by E. coli O157:H7 and Salmonellaenterica on stainless steel surfaces.

In another embodiment, the present invention relates to a method todetect a target substance that is a specific biofilm on a surface. Arepresentative biofilm include films generated by microorganisms such asE. Coli or Salmonella. In this application a characteristic spectrum ofthe specific biofilm is obtained by conventional hyperspectral imagingmethods or other spectral means. Then specific selected wavelengths aredetermined that identify the desired spectra. Filter elements passingthe selected wavelengths are incorporated into the elements of thespectral filter array. A target algorithm is established to reliablydetect the chosen target substance. The present invention acquires andprocesses a number of spectral images as described herein. Finally,custom color displays may be generated to highlight the presence of agiven biofilm on a surface within the field of view of the system.

In the above manner, the present invention may be used as part of asanitation process for inanimate surfaces to identify the presence ofbiofilms before, during or after sanitation of a surface in areas suchas food services, food processing, meat processing, hospitals, clinicsor any location where contamination by microorganisms is a concern.

In the above manner, the present invention may also be used to identifythe presence of biofilms on biological surfaces such as skin, dentalsurfaces, intestinal surfaces, endoarterial surfaces or any otherbiological surface.

Numerous other surfaces and biofilm microorganisms are envisioned,beyond the examples above, in the application of the present inventionand are hereby included.

Hand Washing—Food Safety

Hand washing is a vital component for any food safety program in suchplaces as restaurants, fast food facilities, food preparationbusinesses, and food processing facilities. Historically, compliance waslargely dependent upon training and the integrity of the individualemployee. Even the use of food processing gloves does not assuresanitation unless the gloves are changed at appropriate intervals toavoid passing contaminants on the outside of the gloves. Recently, videocameras have been added as an enforcement tool which can confirm handwashing actions, but not results.

In another aspect, the present invention relates to a method to improvehand washing efficacy as part of a food safety program. In this role,the present invention may be designed to directly inspect the hands andalarm if contaminants are detected. In this manner, the presentinvention becomes a direct measurement tool to detect contaminants thatimpact food safety, strengthening the present strategies used for foodsafety.

Hand Washing—Infection Control

Hand washing is also a vital component for infection control programs inhospitals and medical clinics. Presently, compliance is largelydependent upon the training and the integrity of the individual medicalemployee.

In another aspect, the present invention relates to a method to improvehand washing efficacy as part of an infection control program. In thisrole, the present invention may be designed to directly inspect thehands and alarm if contaminants are detected. Common contaminants thatare handled by medical personnel include blood, feces, urine, and bodilyfluids, which are all readily detectible via the present invention. Inthis manner, the present invention becomes a direct measurement tool todetect contaminants that impact the spread of infection within a medicalfacility.

Egg Inspection

U.S. annual production of table eggs exceeds 90 billion eggs. Whileblood spots appear in less than 1%, this still results in many millionsof eggs containing blood spots. In hatching eggs, the early detection offertilization is important in order to remove unfertilized eggs that mayharbor bacterial growth and contaminate the hatchery. Cracked eggs thatenable bacteria to enter the egg are a concern for both table eggs andhatching eggs, including the spread of Salmonella to human subjects.

Conventional hyperspectral imaging systems have recently been used toinspect eggs for defects. Hyperspectral imaging has been shown to bevaluable in screening eggs for early fertilization detection (0-3 days),blood spots, bad egg (bacteria), and air sack size using a hyperspectralcamera sensitive to wavelengths from 400-900 nm. It was found not to beaccurate for crack detection. “The hyperspectral imaging system appearscapable of detecting developing hatching eggs by Day 3 of incubation ata 91% rate for white shell eggs, and an 83% rate for brown shell eggs.Blood spots in table eggs can readily be detected with at least a 90%accuracy rate. Cracked shells are difficult for the hyperspectral systemto detect under typical conditions.” D. P. Smith, K. C. Lawrence, and G.W. Heitschmidt, Detection of hatching and table egg defects usinghyperspectral imaging, USDA, Agricultural Research Service, RussellResearch Center, Athens, Ga., USA,www.cabi.org/animalscience/Uploads/File/AnimalScience/additionalFiles/WPSAVerona/10938.pdf.

The application of the present invention to egg inspection offerssignificant advantages over the conventional hyperspectral imagingsystems. The cost of a conventional hyperspectral system, approximately$150,000, may be reduced to under $5000 using principles of the presentinvention. The present invention is also capable of vastly improvedresolution over a conventional hyperspectral imaging system. Theimproved resolution of the present invention is likely to improve thediagnostic performance for detecting the small, dendritic shape of bloodspots within the egg and may even be capable of detecting cracks in theshell, a critical function that the conventional hyperspectral systemcannot achieve. The size, mobility and weight of the present inventionare also key advantages over the conventional hyperspectral imagingsystem.

In one aspect, the present invention relates to a method to inspecttable eggs for defects such as blood spots, bacterial contamination andcracks.

In another aspect, the present invention relates to a method to evaluatehatching eggs for fertility and infertility, bacterial contamination,proper development, and cracks.

In another aspect, the present invention relates to a method to inspecteggs for fecal contamination on the surface of the shell since avianfeces may contain Salmonella and E. Coli. The presence of feces on theegg surface may be a serious contaminant. In this aspect, the spectra offeces and the spectra of egg shell are analyzed to select thewavelengths to best discriminate between the two materials. The filterelements are chosen to pass the s wavelengths and the multi-cameraimages are analyzed to determine the presence of fecal contamination ina manner described herein.

In another aspect, the present invention relates to a method to evaluatethe health of the embryo at various stages of development by applying anaspect of the medical imaging diagnostics described elsewhere in thisspecification.

In another aspect, the present invention relates to a remote,noninvasive means to evaluate the chemical composition within the egg byevaluating the spectral composition of light passing through the egg. Inthe case of a fertilized egg, the chemical composition may be spatiallyassigned to anatomical portions of the developing embryo in order toassess a health condition of the developing embryo. In the case of theunfertilized egg, this chemical composition may be used to assess thenutritional value of the egg. Additionally, the different chemicalcompositions of the air sack, albumin and yolk permits an estimation ofvolumes based on the spatial positions of these characteristics.

Bird Vision

Scientists have recently determined that many species of birds have nearultraviolet (NUV) vision receptors. J. Rajchard, Ultraviolet (UV) LightPerception by Birds: A Review, Veterinarni Medicina, 54, 2009 (8):351-359. Humans have three types of cone photoreceptors covering thewavelengths of 400-700 nanometers, with occular media (cornea, lens)absorbing UV light, thus shielding the human retina from UV radiation.Many birds on the other hand have no occular absorption of NUV lightand, in fact, have been found to possess 4-5 cone photoreceptors,including one type sensitive to NUV light in the range of 320-400 nm.

Avian subjects have been found to use their NUV vision in a number ofways. While in flight a kestrel can quickly identify areas with highrodent density by observing the UV light reflected from the frequenturine markings along vole (field mouse) trails. Crows have been found toprefer berries with high UV contrast against background. The zebra finchselects seeds based on NUV intensity. Numerous avian species have beenfound to select a mate based on the brightest plumage and markings, asviewed with NUV sensitivity. While some species, such as starlings,crows, and grackles, exhibit little male/female difference in plumagecolor as viewed with human vision, the male and female have remarkablydistinct plumage color and markings when viewed with NUV vision. Thebeaks of juvenile (not adult) gentoo penguins are UV-reflecting, apossible aid to parents when feeding. The UV reflectance ofsupra-orbital combs on red grouse have been found to be an excellentindicator of health since this reflectance is greatly reduced in thepresence of parasitic worms, though no change in comb size or conditionis observable within the human range of vision. Birds also use their UVperception for recognition of their eggs. Similar eggs, as viewed withhuman vision, have remarkably distinct coloration and markings whenviewed under UV light.

In one aspect, the present invention relates to a method to distinguishfeatures of an avian subject using multi-camera/filter elementssensitive to ultraviolet (UV) or near ultraviolet (NUV) portions of theelectromagnetic spectrum.

In another aspect, the present invention relates to a method todistinguish features of an avian subject using multi-camera/filterelements sensitive to any portion of the electromagnetic spectrum.

In another aspect, the present invention relates to a method todistinguish the sex of an avian subject based upon the ability todistinguish between male and female markings made visible by examiningdifferent parts of the electromagnetic spectrum.

In another aspect, the present invention relates to a method todistinguish the sex of a recently hatched avian or poultry subject,based upon the ability to distinguish between male and female markingsmade visible by examining different parts of the electromagneticspectrum.

In another aspect, the present invention relates to a method todetermine the sex of a recently hatched avian or poultry subject, basedupon the ability to distinguish between male and female markings madevisible by examining different parts of the electromagnetic spectrum. Inthis aspect, the present invention may be part of a machine visionsystem which detects the sex of the avian subject and enables a sortingstep to separate male and female subjects.

In another aspect, the present invention relates to a method todetermine the health of an avian or poultry subject, by examiningdifferent parts of avian anatomy using various portions of theelectromagnetic spectrum.

Urine Detection

Urine detection is valuable in numerous fields. As an example, sincerodents leave frequent urine markings, the present invention may be usedby pest control experts to identify rodent trails and entry points aspart of a pest control program. Hunters may use the present device in asimilar manner to easily identify game trails and marking sites. Subtledifferences in the spectra of urine samples may be useful to medical andveterinary technicians to detect chemical substances in the body.

In another aspect, the present invention relates to a method to deriveinformation about urine within the target field of view. For example, ananalysis of the urine absorption and reflectance spectra may be used toselect a suitable number of filter elements and wavelengths to implementvia an interchangeable filter array. In this application, suitablewavelengths are likely to be in the range between 200 nm and 400 nm.

In another aspect, the present invention relates to a method to analyzeurine samples for medical and veterinary purposes.

Diabetic Foot Perfusion

Diabetic foot disease is a formidable problem due to both its complexityand the increasing diabetic population. Diabetes is the most commoncause of non-traumatic, lower limb amputation with diabetics over 20times more likely to undergo amputation than the general population. Akey component of the disease is compromised peripheral perfusion. Theimaging capabilities of the present invention described herein provide avaluable diagnostic tool in the battle against diabetic foot disease.

Diabetic Foot Vascular Imaging

In one aspect, the present invention relates to a method to image thehealth of the peripheral vessels of the lower limbs. The vascularimaging capabilities of the present invention described herein are apowerful tool which may be used to identify compromised arteries andveins of the lower limbs. Used in the reflectance mode, superficialvessels may be inspected. Used in the transmission mode, bothsuperficial and deep vessels may be examined. In this aspect, theselected wavelengths are chosen to differentiate between arterial andvenous blood. Vascular images and trees may be achieved for lower limbcirculation. The low cost, small size, and mobility, coupled with thehigh-resolution imaging output, provide a device which can be availablein physician offices and clinics. The widespread availability of thispowerful technology, in mobile or stationary form, makes it a formidabletool.

Diabetic Foot Wound Diagnostics

In another aspect, the present invention relates to a method to imagethe health of tissue surrounding an ulcer or wound on the foot. Thehealing ability of an ulcer or wound is largely dependent upon the bloodsupply available. In this aspect, the selected wavelengths are chosen todifferentiate between oxyhemoglobin and deoxyhemoglobin in the tissuesurrounding the wound, much as pulse oximetry is used to determine theoxygen saturation of a finger. However, the present invention providesthe ability to obtain and display a spatial map of this tissueoxygenation (rather than a single average measurement as in the case ofpulse oximetry). Such a tool is useful both for an immediate evaluationof tissue perfusion surrounding the wound, and also as a benchmarkcomparison over a longer time period to evaluate progress of woundhealing.

For example, FIG. 17 shows a spectral image at 600 nm of a human retinashowing arterioles and venules. The image at this wavelength shows thedifferent vascular contrast due to the different absorption spectrum ofhemoglobin varying according to oxygen saturation. Consequently, thearterioles and venules are easily distinguished. These images also maybe color-coded. For example, arterioles could be shown as red, whilevenules are shown as blue.

In another aspect, the present invention relates to a method to providea low-cost, mobile, high-resolution imaging system that may be madeavailable to patients in physician offices, clinics and even as acapability of a personal smartphone or computing device. Stationarydevices are also an option.

Peripheral Vascular Disease

General Peripheral Vascular Disease

In another aspect, the present invention relates to a method to diagnoseperipheral vascular disease in general. The methods described for thediabetic foot may be expanded to include other peripheral vessels.

Feedback for Procedures Intended to Open Occluded Peripheral Vessels

In another aspect, the present invention relates to a method to provideimmediate feedback for procedures intended to open occluded peripheralvessels. In this aspect the present invention may be used to spatiallymonitor tissue oxygenation within the field of view. A series of images,taken during the vascular procedure, can provide an efficacy record forthe procedure. Provided as real-time still images, or a real-time videoimage, this feedback can be a valuable part of the procedure. As theoccluded vessel is opened during the procedure, tissue oxygenation fedby the enhanced blood flow changes and may be viewed with the presentinvention. The change in tissue oxygenation may be used as an indicatorof vessel occlusion during the procedure and in follow-up visitsthereafter.

Wound Healing

A vital component in wound healing is the oxygenation of tissuesurrounding the wound, which is a function of the gradualre-establishment of small blood vessels disrupted by the injury.Presently, this process cannot be accurately assessed without biopsiesor transcutaneous techniques limited to a single point.

In one aspect, the present invention relates to a method to assess woundhealing. The methods described for the diabetic foot and peripheralvascular disease may be expanded to include other wounds as well. Theability to quantitatively obtain a spatial oxygenation level, based onthe spectra of oxygenated and deoxygenated hemoglobin, provides animmediately valuable diagnostic tool for clinicians. The ability totrack changes in these spatial oxygenation images over time provides aquantitative measure of the healing progression that can identify normalhealing as well as problems that may arise.

Feedback for Cardiopulmonary Resuscitation (CPR)

During cardiopulmonary resuscitation (CPR) a primary goal is to maintainbrain oxygenation at a level sufficient to avoid brain injury or death.The techniques, methods, equipment, rescue personnel and patientcondition are often less than ideal due to the emergency nature of theevent. While the compression rate, compression duration, placement ofthe hands, and depth of compression are all critical factors in thesuccessful administration of CPR, there is often little feedback to therescuer regarding the efficacy of their technique. Throughout a CPRevent, conditions change and the rescuer may become fatigued. Thepresent invention offers immediate feedback to the rescuer during CPR,enabling the rescuer to optimize the technique based upon an immediatephysiological measure of tissue perfusion resulting from the CPRmethods.

In one aspect, the present invention relates to a method to provideefficacy feedback to a rescuer administering CPR. As a preferredembodiment in a human subject, the common carotid arteries provide bloodflow to the brain via the internal carotid arteries and the circle ofWillis, as well as blood flow to skin tissue in the forehead region viathe external carotid and temporal arteries. Thus, the oxygenation levelof forehead tissue may be used as an indicator of carotid blood flowduring CPR and an indirect indicator of brain perfusion. Therefore, bymonitoring the tissue oxygenation levels in the forehead region via thepresent invention, using the oxyhemoglobin and deoxyhemoglobin spectradescribed herein, a real-time, indication of brain perfusion during CPRmay be obtained. Using the present invention in this manner, the rescuermay observe the perfusion effect resulting from any change in CPRtechnique, such as hand placement, depth of compression, compressionrate, compression duration, or the like. Such real-time feedback permitsa rescuer to monitor and adjust CPR technique to optimize carotid bloodflow during a rescue session using the present invention aimed at aconvenient, exposed surface on a patient, such as the forehead region.Other vascular beds may also be monitored as an indication of perfusion.

Cardiovascular Risk Assessment Via a Reactive Hyperemia Protocol

Cardiovascular disease is the leading cause of death in the UnitedStates and many developed countries. Non-invasive, low-cost methods aresought to screen asymptomatic, at-risk patients in order to identifythose individuals who should undergo more sophisticated and expensivediagnostic tests to determine necessary treatments such as stents,systemic drug therapy, or vascular surgery. Present screening tests mayinclude a transient brachial artery occlusion via an inflated cuff,followed by a reperfusion protocol, to measure reactive hyperemia. It isthought that the ischemia created by the temporary occlusion of theproximal artery triggers the release of nitric oxide (NO) from healthyendothelial cells within the arterial wall which in turn triggers adistal dilation of the microvasculature. The magnitude of the dilationresponse is an indication of the health of the endothelial cells withinthe artery. The release of nitric oxide is indirectly detected andmeasured using a number of sensors, including digital thermal monitoring(fingertip temperature), fingertip pressure, laser Doppler (skinperfusion), ultrasonic Doppler blood flow (radial or ulnar arteries),and skin perfusion pressure.

In one aspect, the present invention relates to a method to directlymeasure the release of nitric oxide (NO) during a reactive hyperemiaprotocol, by selecting wavelengths that are sensitive to the presence ofNO to include via the filter array. In this manner, a cuffinflation/deflation protocol may be conducted while simultaneouslyhaving the distal region of the limb within the field of view of thepresent invention, thus recording a sequence of images that indicate thechanging concentration of NO at the distal target site. The timing andmagnitude of the nitric oxide curve detected at the distal site isindicative of the arterial reactivity and the health of the arterialsystem. The quantitative health of the arterial system is a strongindicator of a patient's cardiovascular risk.

In another aspect, the present invention relates to a method to measurethe dilation within the capillary beds distal to the cuff during theocclusion/reperfusion protocol. In this aspect, the wavelengths of thefilter array are selected to be sensitive to changes in theoxyhemoglobin and deoxyhemoglobin spectra as described elsewhere withinthis specification.

In this manner, the cuff inflation/deflation protocol is conducted whilesimultaneously having the distal region of the limb within the field ofview of the present invention, recording a sequence of images thatindicate the changing concentrations of oxyhemoglobin anddeoxyhemoglobin at the distal target site. The timing and magnitude ofthe oxygenation curve within the blood, detected at the distal site, isindicative of the arterial reactivity and the health of the arterialsystem. The quantitative health of the arterial system is a strongindicator of a patient's cardiovascular risk.

Surgical Tool to Indicate Vascular Clamping

Researchers demonstrated the use of a modified hyperspectral imagingsystem to examine the spectral characteristics of oxyhemoglobin anddeoxyhemoglobin to aid surgeons by monitoring the oxygenation status oftissue. Karel Zuzak, Robert Francis, Jack Smith, Chad Tracy, JeffreyCadeddu, and Edward Livingston, Novel Hyperspectral Imager AidsSurgeons, SPIE Newsroom, 2008, 0.1117/2.1200812.1394. In one aspect, thepresent invention also would be useful to monitor the spectralcharacteristics of blood components, such as, oxyhemoglobin anddeoxyhemoglobin. Advantageously, the present invention provides areal-time, sequence of still or video images showing conditions ofischemia for tissue and organs that have had their blood supplyrestricted due to vascular clamping during surgery. In many modes ofpractice, spectral imaging systems of the present invention are small,light-weight, and economical. Additionally, the present inventionprovides higher resolution than many conventional hyperspectral imagingsystems and can operate at a video rate, 30 images per second.

In another aspect, the present invention relates to a method to monitorand display the length of time that a given organ or region of tissuehas experienced ischemia. This is accomplished by monitoring regions ofthe field of view and detecting a transition from an oxygenated to adeoxygenated condition. Additionally, the reverse transition is alsodetected. Upon detecting the former transition, “ischemic”, a timer isinitiated. When the latter is detected for the same region, “normal”,the timer is stopped and the elapsed time recorded in memory. Ifmultiple transitions between “ischemic” and “normal” are detected,displays may indicate: a) current ischemic time, b) cumulative ischemictime, and c) the number of times the target region was ischemic.

Lymphatic Imaging

Sentinel lymph node (SLN) mapping is currently the standard of care forstaging breast cancer. J. Sven D. Mieog, Susan L. Troyan, MerlijnHutteman, Kevin J. Donohoe, Joost R. van der Vorst, Alan Stockdale,Gerrit-Jan Liefers, Hak Soo Choi, Summer L. Gibbs-Strauss, Hein Putter,Sylvain Gioux, Peter J. K. Kuppen, Yoshitomo Ashitate, Clemens W. G. M.Löwik, Vincent T. H. B. M. Smit, Rafiou Oketokoun, Long H. Ngo, CornelisJ. H. van de Velde, John V. Frangioni, and Alexander L. Vahrmeijer;Toward Optimization of Imaging System and Lymphatic Tracer forNear-Infrared Fluorescent Sentinel Lymph Node Mapping in Breast Cancer;Annals of Surgical Oncology, 201110.1245/s10434-011-1566-x. Typically, acombination of radioactive colloid and blue dye is injected into thelymphatic system to provide the necessary contrast. However, thisexposes the patients and caregivers to ionizing radiation and blue dyescannot be seen through skin and fatty tissue. Recent clinical dataindicate that near-infrared (NIR) fluorescence imaging, using the NIRfluorescence agent indocyanine green (ICG), enables real-timetranscutaneous visualization of lymphatic channels and detection of SLNwithout the risk of ionizing radiation to patient and caregivers. Thefirst generation of equipment, FLARE, to utilize this approach wasdeemed too large and cumbersome to distribute to clinicians. The secondgeneration, mini-FLARE, reduced the weight to 272 pounds, including a 95lb. arm and an 8.8 lb. imaging head. This mini-FLARE system is stilllarge, heavy and expensive. While the shortcomings of the imaging systemare significant, the images obtained are powerful and provide theclinician with diagnostic information previously available only withradioactive colloid. The present invention provides an alternative wayto achieve the sentinel lymph node mapping using NIR fluorescent dye (noradioactive colloid required) without the expensive FLARE (heavy,unspecified weight) or mini-FLARE (smaller, 272 lbs.) equipment.

In one aspect, the present invention relates to a method to providesentinel lymph node mapping using NIR fluorescent agents in place of thecommonly used radioactive colloid and blue dye. With an economical,handheld size and weight of under one pound, the present inventionprovides a means to achieve the desired SLN mapping using fluorescentdye without the significant disadvantages of the cumbersome andexpensive, mini-FLARE, system that weighs 272 lbs.

Lymph Node Surgery

Researchers demonstrated that an NIR fluorescence detection system ishelpful to surgeons in the surgical removal of a fluorescence-labelledlymph node from the neck of a rabbit. Heuveling D A, Visser G W, deGroot M, de Boer J F, Baclayon M, Roos W H, Wuite G J, Leemans C R, deBree R, van Dongen G A—Eur. J. Nucl. Med. Mol. Imaging (2012). Undernormal surgical lights, the boundaries of a lymph node are not welldefined. However, with fluorescent labeling and lighting the boundariesare easier to see.

In one aspect, the present invention may be used for NIR fluorescent dyedetection to aid in the surgical removal of lymph nodes. Dye is deployedin a patient in a conventional manner such that lymph node tissuesselectively incorporate the dye. A system of the present invention canbe used to capture images of the patient through filters that allow thelymph nodes to be distinguished from other tissues. Analysis of thecaptured images allows the lymph nodes to be identified and located withhigh precision. For example, the present invention can provide acolor-coded image showing the boundaries of lymph nodes as color-codedhighlights layered on a color or grayscale spatial reference image. Thiscolor-coded image would provide significant definition of the lymph nodeboundaries—a critical component in surgical removal of the node. Thefluorescent dye coupled with the principles of the present inventionprovide the high contrast necessary to enhance the boundaries of thelymph node to aid in surgical removal.

Peripheral Artery Disease (PAD)

Peripheral artery disease (PAD) is a disease of the circulatory systemcharacterized by reduced flow to limbs due to a narrowing or blockage ofthe arteries.

In one aspect, the present invention relates to a method to diagnose andmonitor treatment of PAD when the present invention is used in one ormore of the following ways that have been previously described:

-   -   a) to evaluate a patient's cardiovascular risk via a reactive        hyperemia protocol, such that an indication of the health of the        endothelial cells, lining the walls of an artery, is determined;    -   b) to evaluate the oxygenation of a tissue region distal to an        artery, similar to evaluating the oxygenation near a wound, in        order to evaluate the peripheral perfusion effects of PAD;    -   c) to image the vascular tree in the periphery, such as was        described previously, to identify narrowing or occlusion of the        vessels via a non-invasive, economical technique;    -   d) to monitor the progress and effect of an invasive protocol        such as balloon angioplasty, endarterectomy (i.e., scalpel,        ultrasound catheter, orbital atherectomy) both during the        procedure and during the weeks and months following the        procedure.        Breast Cancer Detection

Researchers presented absorption spectra for a 30-mm diameter tumor inthe breast of a chemotherapy subject and spectra for normal breasttissue that shows a significant spectral difference between normaltissue and a tumor. Cerussi A, Hsiang, D., Shah, N., Mehta, R., Durkin,A., Butler, J Tromberg, B., Predicting Response to Breast CancerNeoadjuvant Chemotherapy Using Diffuse Optical Spectrscopy, Proc. Natl.Acad. Sci. U.S.A, 104: 4014-4019, 2007. The heightened absorption forthe tumor, as shown in FIG. 2E, is a result of increased hemoglobin andwater in the region of the tumor with respect to normal tissue.

In one aspect, the present invention relates to a method to detect tumortissue in a breast as compared to normal breast tissue. In such anapplication, the heightened absorption in the region of the tumor, dueto the increase in hemoglobin and water, can be differentiated from thatof normal breast tissue. In such an application, the selectedwavelengths determined by an analysis of the respective spectra, will besensitive to the spectral differences of the two tissue types asdisplayed in FIG. 2E.

Skin Cancer Detection

As there are different spectral characteristics for tumor and normalbreast tissues, it is anticipated that there are similar spectraldifferences for cancerous and normal skin tissues.

In another aspect, the present invention relates to a method to detectskin cancer by detecting the spectral differences between the spectra ofcancerous and normal skin tissues in a manner consistent with thedescription of the present invention.

Medical Imaging

Medical applications of spectral imaging using ultraviolet, visible,near infrared and infrared imaging methods are numerous. Examples ofthese applications include blood flow detection, tissue perfusion,location of arteries and veins, urine tests, skin diagnostics, estrusdetection, tumor detection, lymphatic system imaging, and lymph nodesurgery. These methods may be used using the spectral characteristics ofblood, tissue, and bodily fluids independently and in conjunction withfluorescent dyes. To date most of these applications require dedicatedequipment that is large, expensive and immobile. There is great need forthe present invention in these applications, providing an economicalsolution with improved performance and greater mobility. Even smallstationary devices offer great advantage over the large, cumbersome sizeof many medical imaging systems.

In one aspect, the present invention is related to a method to provide amedical image of a patient using one or more of the spectralcharacteristics present within the target field of view and the methodsof the present invention described herein.

In another aspect, the present invention is related to a common medicalimaging platform which may be used for different specific imagingapplications by simply interchanging a filter card and loading theappropriate algorithm from electronic memory. The present inventionpermits a hospital or clinic to cost effectively meet numerous medicalimaging needs with a common instrument platform and a library of filtercards (and associated software algorithms) instead of expensive,dedicated imaging systems for each application.

Medical Diagnostic Information Automatically Linked to an IndividualSubject

In another aspect, the present invention relates to a method to acquireand analyze multi-camera data of an individual animal or human within atarget field of view, simultaneously acquiring a unique individualidentification, and entering the multi-camera data into a databaseorganized according to the individual human or animal. The individualidentification may be, but is not limited to, a radio frequencyidentification (RFID), bar code, patient identification bracelet, manualidentification number, or hospital patient record number.

Veterinary and Human Applications

Since animal physiology and human physiology are in many ways similar oridentical, the medical applications described herein also apply tosimilar veterinary applications. Likewise, veterinary applications mayreadily be adapted for human use. Veterinary applications include, butare not limited to, mammalian applications for small animals, livestock,and companion animals, aquatic animals, as well as avian or poultryapplications. The common physiological characteristics permit auniversal application across many species.

In one aspect, the present invention relates to a method to apply novelspectral imaging methods to both animals and humans utilizing theirsimilar anatomical and physiological characteristics.

Estrus Detection

Dairy Management

The accurate and efficient detection of estrus for dairy cows isessential for a quality breeding program and the economical managementof a dairy herd. The most widely used indicator of estrus, or heat, isthe manual observation of when a cow permits herd mates to mount whileshe remains standing. This surveillance of mounting activity islabor-intensive and expensive. Present precision dairy technologies relylargely on behavior-based, indirect indicators of estrus such asrumination detectors (narrow-band microphones) and activity sensors(accelerometers).

One direct, physiological sign of estrus is a blood-swollen vulva,triggered by the increase in estrogen at the onset of estrus. A vaginal,mucus discharge is also common and frequently smears on the rump andtail. Chemical changes, due to hormonal stimulation, occur in bodilyfluids such as urine, blood, vaginal mucus, saliva, nasal discharge,feces and tears.

In one aspect, the present invention relates to a method to indicateestrus by detecting an increase in blood flow to the vulva by sensingthe oxygenation level of vulvar tissue in a manner similar to thatdescribed for observing tissue oxygenation for the diabetic foot or thetissue surrounding a wound. This method uses wavelengths sensitive tooxyhemoglobin and deoxyhemoglobin to evaluate tissue oxygenation. Athreshold may be established to differentiate between vaginal tissueoxygenation levels during estrus and vaginal tissue conditions at othertimes during a reproductive cycle.

In another aspect, the present invention relates to a method to indicateestrus by detecting chemical changes in bodily fluids, such as urine,blood, vaginal mucus, saliva, nasal discharge, feces and tears. Thismethod uses wavelengths sensitive to the specific chemical change of agiven bodily fluid during estrus.

In another aspect, the present invention relates to a method which maybe used in a milking parlor or feed area to automatically scanindividual cows each time they come into the area. Reproductive dataacquired in this manner is available for inclusion in an animal database with the identify of the individual animals detected by automatedanimal identification methods, such as RFID, or manually entered intothe system. In this manner, statistical analysis of these reproductivedata may be used to closely track the reproductive cycle of anindividual cow to predict the optimal time for breeding.

In another aspect, the present invention relates to a method to providea portable, hand-held device which may be used by dairy managers tomanually scan individual cows anywhere on the farm and at any time.Reproductive data acquired in this manner is also available forinclusion in a dairy data base. Individual cows may be identified byeither automated or manual identification methods. Statistical analysisof these reproductive data may be used to closely track the reproductivecycle of an individual cow to predict the optimal time for breeding.

In another aspect, the present invention relates to a method todetermine the health of an individual cow. The reproductive data andchemical analysis of bodily fluids, acquired in the manners describedabove, may be further processed to detect trends and variations whichare indicative of the health of the individual animal. The combinedhealth of individual animals may be collectively analyzed to monitor thehealth of the entire herd.

Livestock Management

The detection of estrus as part of a livestock breeding program iscritical. As with dairy cows, (described above) the detection of estrusin other livestock, such as beef cattle, pigs, goats, sheep, horses,bison, deer, elk, camels, llamas, and donkeys, is often accomplished,with varying degrees of efficiency, by labor-intensive observation. Thedairy applications of the present invention, described above, may bereadily adapted for any number of other animal species.

Forensics and Criminology

In another aspect, the present invention relates to a method to detectforensic substances such as bodily fluids, fingerprints, gunshotresidue, and bruise enhancement. Spectra of the desired target substanceand anticipated background substances are analyzed and criticalwavelengths selected. Filter elements are chosen according to theselected wavelengths. The present invention may be particularlyadvantageous in outdoor settings where present technologies requirelight-blocking structures and expensive, narrowband, bulky, lightsources, since the present invention may detect the target substanceusing natural, broad-spectrum sunlight. Additionally the presentinvention is advantageous since it can directly detect forensicsubstances without the use of fluorescent sprays, such as Luminol forblood detection, which may dilute or otherwise degrade the forensicsubstance of interest.

In another aspect, the present invention relates to a method to combinethe filter elements necessary to detect multiple forensic targetsubstances in a single filter card (as an alternative, electronictunable filters may be used). For example, a single filter card maycontain all the necessary filter elements for selected wavelengthsnecessary to detect blood, semen, and gunshot residue. When using such acombination filter card, program instructions are combined to providethe algorithm to detect and display each target substance. As anexample, a single grayscale, spatial orientation image of a crime scenemay be displayed with blood highlighted in red, semen highlighted inwhite, and gunshot resident highlighted in blue. Obviously, other colorschemes are possible and are included, herein.

Building Inspection

Building inspections presently rely highly on visible observations orthermal cameras (long wave infrared, 8 μm to 14 μm). While thermalcameras provide an indication of moisture due to evaporative cooling,they lack the ability to indicate damage that has achieved the sametemperature as its surrounding. Thermal cameras also lack a capabilityfor chemical analysis, are low-resolution and typically lack theconnectivity, via internet or cell phone system, of the presentinvention.

In another aspect, the present invention relates to a method to detectbuilding damage via chemical signatures characteristic of moisture,mold, mildew, smoke, pet urine, and pest infestation (rodents,cockroaches, termites, scorpions, bedbugs, spiders, stink bugs,carpenter ants, etc.). Each of these defects is characterized bychemical substances within the building that may be invisible to thehuman eye. Spectra of the desired target substance and anticipatedbackground substances are analyzed and wavelengths selected. Filterelements are selected according to the selected wavelengths as describedpreviously. As an example, a urine reflectance spectrum is characterizedby a reflectance peak in the ultraviolet range near 375 nm with minimalreflectance around 450 nm. These urine characteristics may be used withthe present invention to view pet urine stains on walls and carpet,rodent trails highlighted by urine markings, insect trails indicated byurine marks visible with the present invention, but not the unaidedhuman eye. Similarly, the present invention may be used to detect thecharacteristic, vegetative spectra of mold and mildew where onlyinanimate surfaces should exist. Walls infested with rodents, carpenterants, or termites may exhibit a combination of urine and moisturespectra.

Floor Cleaning

In another aspect, the present invention relates to a method to detectpet urine, mildew, smoke, mold or other contaminant on floors andcarpets as part of a cleaning procedure or operation.

Pest Control

Pest control experts face a multitude of pest species and habitats withfew diagnostic tools. Optical assistance is typically limited to anultraviolet, fluorescent tube (‘black light wand’) and perhaps orangegoggles (to filter out excessive UV light which may obscure the targetand damage the eyes).

In another aspect, the present invention is related to a method todetect pest signatures. For example, the present invention may be usedtrack rodent urine trails, identify fluorescent scorpions (UV), anddetect blood residue from a bed bug infestation under normal outdoor orincandescent light conditions. The characteristics described hereinregarding building inspection damage may be used to identify, track andexterminate pests as well as indicate the effectiveness of suchtreatment. Additionally, the smartphone interface of the presentinvention provides a convenient means to immediately communicate aninstantaneous chemical analysis of an infestation scene to companyexperts for review and guidance.

Automotive

Leaks in automotive systems are commonly detected by placing ultravioletdyes in the liquid or gas such that the leak fluoresces when exposed toultraviolet light. In this manner a leaking substance, such as coolant,freon, engine oil, or brake fluid may be readily detected. Presentlysuch ultraviolet dyes require low light conditions and a black lightwand.

In another aspect, the present invention is related to a method to viewthe spatial distribution of ultraviolet dyes which provide a mapping ofthe automotive system leaks. Using the present invention these leaktracers may be viewed under normal daylight conditions, incandescentlighting or custom LED light sources. Since the spectral characteristicsof a given line of UV dye is readily specified, the necessary filterelements may be provided on a standard filter card and the specificalgorithms stored in a data table for easy access.

In another aspect, the present invention is related to a method to viewleaks in other systems, such as robotic or hydraulic systems, using themethods described above for automotive systems.

Fluorescent Dyes

There is a large and growing market in fluorescent dyes. Applicationsinclude automotive fluid leaks, oil pipeline leaks, biophotonics(medical), nano technologies, industrial tracking and many others. Onesource, Crysta-Lyn Chemical Company, advertises over 800 “proprietaryNIR, fluorescent, UV and laser dyes.”

In another aspect, the present invention is related to a method to viewfluorescent dyes in numerous applications. The spatial distribution ofthese dyes within the field of view and even their concentration may beprovided with the methods of the present invention. The low-cost, highresolution and daylight viewing of the present invention provide anadvantageous enhancement to the use of these fluorescent dyes.

Food Inspection and Processing

Some applications of hyperspectral imaging are coming to fruition in thefood industry. The capability to obtain a real-time chemical analysis offood products on a conveyor belt is powerful. However, the manyshortcomings of conventional hyperspectral imaging, such as high cost($100,000-$200,000 per camera system), low resolution, linear scanningand large installation size, remain constant obstacles to widespread usewithin the food industry. A strong need exists for an economical imagingsystem that provides many of the benefits of hyperspectral imaging whileovercoming the shortcomings.

In one aspect, the present invention is related to a method to inspectfoods within the food industry. The target spectra of individual foodsmay be acquired as described herein. These spectra are then analyzed toselect the wavelengths necessary to detect or measure the desired foodproperty. The elements of the filter array are chosen to pass theselected wavelengths such that each image of the multi-camera arrayrepresents a different wavelength image. These images are then alignedand analyzed to detect or measure the desired food property as describedherein. Examples of food industry applications include, but are notlimited to, adulterant screening, fresh produce quality control(bruising, ripeness detection), grain screening, legume screening,screening for feces materials, insects, and bacteria (the biofilmsassociated with some strains of Salmonella and E Coli are UVfluorescent), moisture control, ingredient control, mixture controls,raw component screening, equipment sanitation, and packaging qualitycontrol. Numerous food safety issues can also be addressed with thepresent invention.

Mineral Detection and Processing

Hyperspectral imaging is presently in use for screening and processingminerals. The capability to obtain a real-time chemical analysis ofvarious mineral and ore products on a conveyor belt is powerful.However, the many shortcomings of conventional hyperspectral imaging,such as high cost ($100,000-200,000 per monitoring station), lowresolution, linear scanning and large installation size, remain constantobstacles to widespread use within the mining industry. A strong needexists for an economical imaging system that provides many of thebenefits of conventional hyperspectral imaging while overcoming theshortcomings.

In one aspect, the present invention is related to a method to inspectgems and minerals, such as within the mining industry. The spectra ofindividual gem, mineral and ore properties may be acquired as describedherein. These spectra are then analyzed to select the wavelengthsnecessary to detect or measure the desired mineral property. Theelements of the filter array are chosen to pass the selected wavelengthssuch that each image of the multi-camera array represents a differentwavelength image. These images are then aligned and analyzed to detector measure the desired mineral property as described herein.

Examples of mineral industry applications include, but are not limitedto, rapid analysis of drill cores, with SWIR and LWIR imaging of valuefor the detection of minerals in the feldspar, silica, calcite, garnetand olivine groups. Many mineral deposits are presently identified fromairborne hyperspectral images. Such examples include minerals associatedwith the presence of gold and diamonds. The relationship between oil andnatural gas leaks on the spectra of nearby vegetation is becoming betterunderstood as a tool to detect leaks near pipelines and natural gaswells. The present invention makes use of spectral information andunderstanding available from conventional hyperspectral systems, inorder to provide a tremendous reduction in system cost as well as agreat increase in resolution, thus providing a higher performance systemat a dramatic reduction in cost.

In another aspect, the present invention is related to a method torapidly analyze drill cores using methods described herein.

In another aspect, the present invention is related to a method to sortgems and minerals on a mechanical conveyance, such as a conveyor belt,using methods described herein.

In another aspect, the present invention is related to a method toreplace conventional hyperspectral imaging systems in the mineralindustry with greatly reduced equipment cost and significantly improvedresolution using methods described herein.

In another aspect, the present invention is related to a method todetect pipeline leaks by recognizing changes in the spectra ofvegetation affected by the leaking pipeline product.

Frac Sand

Frac sand is a material greatly needed within the oil and gas industriesto efficiently extract oil and natural gas from new wells using inducedhydraulic fracturing techniques. There is a need to prospect for newfrac sand deposits as well as screen the quality and moisture content ofthe frac sand during the processing stages.

In one aspect, the present invention relates to a method to deriveinformation about frac sand within the target field of view. Frac sand,with a high composition of quartz, exhibits a broad reflectanceamplitude in the spectral range from 400 to 1100 nm. An analysis of thefrac sand reflectance spectra may be used to select a necessary numberof filter elements and wavelengths to implement via the interchangeablefilter array.

In another aspect, the present invention is related to a method toprospect for new frac sand deposits from a ground-based or airborneplatform.

In another aspect, the present invention is related to a method toanalyze the quality and moisture content of frac sand during theprocessing of the sand.

In another aspect, the present invention is related to a method toanalyze the quality and moisture content of all sands types in a mannerdescribed herein for frac sand.

Feed Inspection and Analysis

The present invention is ideally suited for numerous applications in thefeed industry, such as for livestock, poultry, pets, aquatic animals andthe like. Quality assurance, sorting, analysis, incoming inspection ofraw ingredients, mixtures and many more uses are valuable applicationsfor the present invention. Significant value may be realized where thereflectance or absorbance spectra can be characterized and used todifferentiate or measure a feed substance within the target field ofview.

In one aspect, the present invention relates to a method to analyze ordetect a feed substance within the target field of view using themethods described herein.

In another aspect, the present invention relates to a method to analyzefeed to detect mycotoxins and aflatoxins, measure crude protein, analyzenutrients, and detect contaminants, including pesticides.

In another aspect, the present invention relates to a method to analyzefeed to detect antibiotic residue.

Security and Forgery Detection

Genuine U.S. Federal Reserve Notes, such as denominations $5, $10, $50and $100, have vertical security threads that fluoresce a unique colorwhen illuminated with ultraviolet (UV) light. U.S. Treasury checks alsocontain inks that fluoresce under UV light. Secretive chemical markers(“virtual fingerprints”), not fully visible to the naked eye, may beused as anti-counterfeiting, brand protection, and productauthentication measures for a multitude of U.S. and internationalproducts, such as currency, checks, banknotes, cigarettes, alcohol,pharmaceuticals, consumer goods, high-technology equipment, fuels, andenergy products.

In another aspect, the present invention is related to a method to viewthe security threads and security markings on genuine U.S. FederalReserve Notes and U.S. Treasury checks, containing features thatfluoresce under UV light. The present invention may readily be appliedto this application by analyzing the reflectance spectra of suchdocuments, then selecting UV wavelengths that indicate the documentproperties. The filter elements are chosen to generate imagesrepresentative of the selected wavelengths. After alignment, theseimages are analyzed to provide an indication of the genuineness of thedocument under inspection. The advantages of the present inventioninclude a light-weight, mobile instrument that can view such securityfeatures under incandescent or outdoor lighting conditions. Thisapplication may be expanded to detect other types of security markings,anti-counterfeiting measures, brand protection methods, and productauthentication criteria for many different products.

A secretive chemical or physical marker, which may be referred to as ataggant, may be added to materials and products to permit the materialor products to be clearly identified by detection analysis. Suchmarkers, often in the form of microscopic particles, may be added to thematerial at manufacture so as to identify the material and perhaps eventhe brand, formula, concentration and lot number of that material.Product examples that may contain taggants are inks, papers, glass,perfumes, polymer resins and products fabricated therefrom, officialdocuments, money, fertilizer, chemicals, paint and varnish coatings,packaging, tires, composites, combustible or highly reactive materials,pharmaceuticals, alcohol, luxury products, and the like.

In another aspect, the present invention relates to a method to detectand identify specific security markers or taggants based upon theirspectral characteristics. Such markers, comprised of microscopicparticles or otherwise, may be selected to have specific spectralcharacteristics, in either reflective or absorptive modes, such thatthese spectral characteristics may be viewed using the presentinvention. The present invention may readily be applied to theseapplications by analyzing the reflectance spectra of materialscontaining such identifying markers, then selecting wavelengths thatindicate the spectral properties of the markers. The filter elements arechosen to generate images representative of the selected wavelengths.After alignment, these images are analyzed to provide an indication ofpresence and/or quantity of the markers under inspection. The advantagesof the present invention include an economical, light-weight, mobileinstrument that can view such taggant features under artificial,incandescent or outdoor lighting conditions. This application may beexpanded to detect other types of security markers and indicators.

In another aspect, the present invention relates to a method to view andidentify specific taggants placed in inks, paints, or the like, suchthat invisible printing or coded printing which is not fully visible tohuman vision, may be viewed or decoded using the present invention. Withthe present invention, such printing systems could be used to create anynumber of secure and encoded documents, product labels, symbols,security markings and the like. Additionally, once the taggant printingor markings are identified using the present invention, machine visionmethods for pattern recognition, imaging processing and the like mayalso be applied to provide a wide array of security systems andmarkings.

Hunting

In their quest for game, hunters may use the urine marking of theirprey. On occasion hunters also need to follow blood trails of an animal.Additionally, since prey such as deer and avian subjects may be able tosee hunter clothing that reflects ultraviolet (UV) light, hunters have aneed to view the appearance of their clothing with respect to UVvisibility.

In one aspect, the present invention is related to a method to detectand display the urine markings of prey animals so that a hunter maybetter track the prey using the differences between the spectralcharacteristics of urine and the spectral characteristics of backgroundvegetation.

In another aspect, the present invention is related to a method todisplay a blood trail of an injured animal using the differences betweenthe spectral characteristics of blood and the spectral characteristicsof background vegetation.

In another aspect, the present invention is related to a method todisplay a hunter's clothing as it might appear to a prey animal that cansee UV light reflected from the clothing.

Consumer Color Matching

Consumers often desire to precisely match colors. These colors may bepaint colors on surfaces such as walls, ceilings, floors, cars, or anynumber of other objects. Additionally, these colors may involve articlesof clothing such as shirts, coats, hats, scarves, shoes or otherclothing accessories.

In one aspect, the present invention is related to a method to preciselymatch a color, such as paint or clothing, by recording spectral data atselected wavelengths in order to quantitatively describe an unknowncolor which the user desires to match.

Ultraviolet Tattoos and Body Art

Ultraviolet tattoos and body art have become popular. Invisible to thenaked eye, these works of art use paints and dyes which are visible onlyunder UV light.

In one aspect, the present invention is related to a method to matchcolors used for UV tattoos and body art by recording spectral data atselected wavelengths in order to quantitatively describe an unknown UVcolor which the user desires to match.

In another aspect, the present invention is related to a method to viewUV tattoos and body art by recording spectral data at selectedwavelengths in order to detect and display the UV artwork.

Gas Detection

A gas detector is a device which identifies the presence of variousgases within an area. It may be handheld, battery-powered, or part of asafety system. Gas detectors may be used to detect combustible,flammable, or toxic gases. They may be used to detect a specific gas ora combination or mixture of gases. These type of devices are valuable ina multitude of applications, such as oil rigs, gas pipelines, industrialexhaust monitoring, manufacturing, homes, automotive systems,agricultural facilities and medical respiratory systems.

In one aspect, the present invention is related to a method to detect achosen gas within an area. Using the methods described herein, theunique qualities of the gas spectrum may be used to identify thepresence of the chosen gas within the target field of view and displaythe gaseous cloud as a color-coded presence on the output display.

In another aspect, the present invention is related to a method todetect the concentration of a chosen gas within an area. An in-framereference, desirably positioned at a known distance from the imagingsystem, can be used to help calibrate image information to allowquantity information to be derived from the captured image information.For example, spectral properties for a substance may be a function ofthe concentration of the substance. The in-frame reference may includesamples including the substance at known concentrations. Using themethods described herein, the unique qualities of the gas spectrum maybe used to first identify the presence of the gas within the targetfield of view and then display the gaseous cloud as a color-codedpresence on the output display in a manner that coveys the concentrationof the gaseous cloud.

The application of the present invention to gas detection is valuablewith or without the described display.

Petrochemical

Spectral imaging has numerous applications in the petrochemicalindustry. Vegetation changes caused by pipeline leaks have beenpreviously described. Oil spill residue from any source, in water, soilor along shorelines may be detected by spectral variations.

In one aspect, the present invention relates to a method to detect oilresidue in water, soil or along shoreline using the methods describedherein.

Chemical Processing

The present invention is ideally suited for numerous applications in thechemical processing industry. Quality assurance, sorting, analysis,incoming inspection of raw ingredients, mixtures and many more uses arevaluable applications for the present invention. Significant value maybe realized where the reflectance or absorbance spectra can becharacterized and used to differentiate or measure a chemical substancewithin a target field of view.

In one aspect, the present invention relates to a method to analyze ordetect a chemical substance within the target field of view using themethods described herein.

Manufacturing

The present invention is ideally suited for numerous applications inmanufacturing plants. Quality assurance, sorting, analysis, incominginspection of raw ingredients, mixtures, assemblies, product securitymarkings and many more uses are valuable applications of the presentinvention. Significant value may be realized where the reflectance orabsorbance spectra can be characterized and used to differentiate ormeasure a target substance on a surface or within the target field ofview.

In one aspect, the present invention relates to a method to analyze ordetect a target substance within the target field of view using themethods described herein.

Sustainability

In another aspect, the present invention relates to a method to provideindicators for “sustainability” plans prepared for a government entity(i.e., federal, state, city, county, township) or a corporate/businessentity. Such an indicator may involve a measure of water quality, aerialmeasurements of land use, the health of wetlands and forests, as well asthe location of specific species of vegetation. Additionally, thesesustainability indicators may include, but are not limited to, pestinfestation, water pollution, chemical runoff, chemical spills, buildingcode adherence, and adherence to watering bans.

Machine Vision

The present invention is well suited for numerous applications inmachine vision. Objects within the target field of view may beidentified by their chemical composition, as described herein. Thisobject information may then be applied to many machine vision uses.

In one aspect, the present invention relates to a method to identifysubstances within the field of view by their spectral composition in amanner that serves as input into a machine vision system using themethods described herein.

Meat and Poultry Processing

In addition to similar uses described for food processing, meatprocessing applications also include meat quality control and lean meatvs. fat identification and grading. Recently, conventional hyperspectralimaging has been conceptually shown by the USDA to be capable ofdetecting organic material residue on meat processing equipment aftercleaning. Conventional hyperspectral imaging also shows promise as ameans of grading cuts of meat. As with food processing, this practicalapplication of hyperspectral imaging is severely limited by the large,low-resolution, costly equipment essential for hyperspectral imaging.There exists a need for a reduced size, low-cost, imaging system thatovercomes the significant limitation of conventional hyperspectralimaging for these applications.

The present invention can fill this need. FIG. 18 shows spectralinformation for various contaminants that may be present on the surfacesof stainless steel processing equipment. The data is redrawn after: J.Qin, K. Chao, M. S. Kim, S. Kang, B. K. Cho, W. Jun, Detection ofOrganic Residues on Poultry Processing Equipment Surfaces by LED-InducedFluorescent Imaging, Applied Engineering in Agriculture, AmericanSociety of Agricultural and Biological Engineers ISSN 0883-8542, Vol.27(1): 153-161, 2011. http://naldc.nal.usda.gov/download/49285/PDF. Thespectra of the various substances are easily distinguishable, allowingthe present invention to be used to detect contaminants on stainlesssteel. For example, FIG. 19A shows a schematic image of a contaminated,stainless steel surface. FIG. 19B shows how the present invention canoutput an image showing the presence and location of contaminants on thestainless steel surface of FIG. 19A.

In another aspect, the present invention is related to a method to grademeat, implement reliable meat quality control, analyze lean vs. fatratios, and detect organic material residue on meat processingequipment. The spectra of individual meat properties may be acquired asdescribed herein. These spectra are then analyzed to select thewavelengths necessary to detect or measure the desired meat property.The elements of the filter array are chosen to pass the selectedwavelengths such that each image of the multi-camera array represents adifferent wavelength image. These images are then aligned and analyzedto detect or measure the desired meat property. Sample meats which maybenefit from an analysis via the present invention include, but are notlimited to, beef, pork, bison, goat, sheep, lamb, other mammalianspecies, chicken, turkey, other poultry, fish and seafood. Numerous foodsafety issues can also be addressed with the present invention.

In another embodiment, the present invention would be used to detectfecal contamination on carcasses of species such as beef, pork, poultry,lamb and bison. In fact, the present invention may be used to detectfecal contamination on any number of surfaces.

During poultry processing, the cleanliness of the poultry carcasses isessential. Any fecal contamination on the surface of the carcass canlead to food safety problems which may cause illness or death toconsumers and significant financial loss for the poultry processor.Conventional hyperspectral imaging is being investigated as a means forchemical imaging to detect fecal contamination on poultry carcasses. Ina recent study, a large, expensive, line-scan, hyperspectral imagingsystem was found to reliably detect four types of fecal materials(duodenum, ceca, colon and ingesta). Only three wavelengths were usedfor this detection, 517 nm, 565 nm and 802 nm. Moon S. Kim, Shu-I Tu,Kaunglin Chao, Development of real-time line-scan hyperspectral imagingsystem for online agricultural and food product inspection, Proc. SPIE7676, Sensing for Agriculture and Food Quality and Safety II, 76760J (); doi:10.1117/12.850460 From Conference Volume 7676, Sensing forAgriculture and Food Quality and Safety II, Orlando, Fla., Apr. 5, 2010.The fecal detection algorithm was based on dual band ratios of 565nm/517 nm and 802 nm/517 nm followed by thresholding. The spatial imageswere 118 lines×512 spatial pixels, 0.060 megapixels. This system isestimated to cost $150,000 on one side of a carcass per inspection linewith substantial fixturing and complexity.

In one embodiment the present invention is applied to poultry processingand the spectral information derived for the hyperspectral system isused to select the filter array elements of the present invention, forexample, 517 nm, 565 nm and 802 nm. The target algorithm may be based onthe successful algorithm derived for the hyperspectral system, in theabove example, dual band ratios of 565 nm/517 nm and 802 nm/517 nm.However, the spatial resolution is increased from the low-resolution ofthe hyperspectral system, 0.06 megapixels, to the higher resolution ofthe present invention, 8 megapixels or perhaps 16 megapixels, greatlyincreasing the ability to detect small fecal particles that may goundetected with the lower-resolution, hyperspectral imaging system.Additionally, the cost of a system implementing the present invention isestimated to be a small fraction of the cost of a conventionalhyperspectral imaging system. The reduction in size and powerrequirements is also advantageous for the present invention. In ahandheld package, perhaps attached to a smartphone, tablet ormini-tablet computer, the present invention could be a mobile deviceprovided to each meat inspector. Incorporated into the production line,the present invention would provide a great cost-performance improvementover the higher cost, low-resolution hyperspectral imaging system. (Notethat the conventional hyperspectral imaging system is a significantimprovement over current meat inspection practices which are manualinspection or machine vision in the visible spectrum.).

Researchers have reported contaminants that may be present on poultrycarcasses in the field of poultry processing. B. Park, W. R. Windham, K.C. Lawrence, D. P. Smith, Contaminant Classification of PoultryHyperspectral Imagery using a Spectral Angle Mapper Algorithm,Biosystems Engineering, Volume 96, Issue 3, March 2007, Pages 323-333.The present invention provides remote and noninvasive techniques toeasily and accurately detect such contaminants. Spectral informationsuch as that reported by Park et. al. may be used to select appropriatewavelengths and corresponding filter elements to be used in the practiceof the present invention. The resultant filter array and a camera arrayis then used to capture images of the poultry carcasses to be evaluatedfor contamination. For example, FIG. 20 schematically shows schematicsample images of poultry carcasses that are then captured using thesearrays. A first carcass (on the left) is shown to be uncontaminated,while a second carcass (on the right) is shown to be contaminated fromtop to bottom by duodenum (triangle), cecum (square), colon (circle),and ingesta (star). The present invention not only detects thecontaminants but accurately shows where each contaminant is located inthe image of the second carcass. This also is an example of a mode ofpractice in which the same filter array can be used to detect multiple,specific target substances.

Health

When used to assess health conditions, imaging systems of the presentinvention may be used to detect and/or assess health conditions of ahuman or animal subject associated with at least one of arterialdisease, venous disease, blood flow, the perfusion of peripheral tissue,a wound, a diabetic foot, tissue perfusion, wound healing for a diabeticsubject, peripheral vascular disease, urine analysis, a procedureintended to open an artery or vein, the delivery of cardiopulmonaryresuscitation (CPR), an indication of cardiovascular risk via a reactivehyperemia protocol, the degree of vasodilation, vascular clamping duringsurgery, lymphatic imaging, lymph node mapping, lymph node surgery,breast cancer detection, tumor detection, skin cancer detection, andreproductive status.

Crops

Spectral imaging systems of the present invention may be used to assesscrop conditions associated with at least one of moisture content,nitrogen content, chlorophyll content, plant maturity, disease state,insect infestation, fungus infestation, mold content, weed content,harvest readiness, soil condition, and fertilizer effectiveness.

Urine

When used to detect urine in a scene, principles of the presentinvention may be used to assess at least one of determining aquantitative measure of urine characteristics for medical purposes,determining a quantitative measure of urine characteristics forveterinary purposes, determining a quantitative measure of urinecharacteristics to identify the animal species which provided the urinesample, determining a quantitative measure of urine characteristics inorder to identify the specific animal or human subject which providedthe urine sample, tracking individual animals for hunting orconservation, identify urine spots on carpet, and walls or flooring forinspection or cleaning purposes.

Algorithms

When analyzing spectral information from captured images, the spectralimaging systems may at least consider one of a ratio of spectralamplitudes at selected wavelengths; a difference (or sum) of spectralamplitudes at selected wavelengths; a principle component analysis ofthe spectra associated with the target substance; a basis functionrepresentation of the target spectrum and background spectra; aneigenvector analysis of the spectra associated with the targetsubstance; other signal processing and statistical methods familiar tothose skilled in the art of spectral analysis; and a calibrationinvolving an in-frame reference.

Sanitation

Spectral imaging systems of the present invention may be used to assesssanitation conditions associated with at least one of detecting biofilmon a surface, detecting a surface contaminant, hand washing, egginspection, food processing equipment, food packaging, meat processingequipment, meat packaging, restaurant equipment, hospital equipment,surgery equipment, a transportation container, a transportation vehicle,a vehicle transporting food products, a vehicle transporting meatproducts, a vehicle transporting livestock, and a vehicle transportingpoultry.

Eggs

Spectral imaging systems of the present invention may be used to assessegg conditions selected from one or more of the stage of embryonicdevelopment, the health of the embryonic cardiovascular system, a healthcondition of the embryo, egg infertility, bacterial contamination, fecalcontamination of the egg shells, cracks, blood spots, an addled egg, andthe chemical composition of an egg.

Food

Spectral imaging systems of the present invention may be used to assessfood conditions associated with at least one of adulterant screening,quality control of food properties, fresh produce quality control forproperties, such as bruising, moisture content, and ripeness, grainscreening, legume screening, moisture content/control, ingredientcontrol, mixture control, raw component screening, equipment sanitation,packaging quality control, quality control of finished food products,detection of bacterial contamination, and biofilm contamination.

Meat

Spectral imaging systems of the present invention may be used to assessmeat conditions associated with at least one of detecting contaminantson meat or carcass surfaces, detecting organic material residue orcontaminants on processing equipment, determining meat properties, meatgrading, meat quality control, and analysis of lean vs. fat ratios.

Another embodiment of a spectral imaging system 500 is shown in FIG. 21.This embodiment includes advantages such that an interchangeable filtercard can be smaller due to closer spacing among filter elements. Also,the system may have a thinner profile since the optical path from thefilter card to each image capturing element is substantially parallel tothe camera plane.

In more detail, system 500 includes an image capture array of two ormore image capture elements 502 and 503 (two such elements are shown forpurposes of illustration). Image capture element 502 includes a lens504, a camera body 506, and an image sensor 508. Image capture element503 includes a lens 505, a camera body 507, and an image sensor 509.

System 500 further includes a filter card 510 that includes a spectralfilter array including two or more spectral filter elements. Forpurposes of illustration, filter elements 512 and 514 are shown. Filterelement 512 is designed to selectively pass a bandwidth portion of theelectromagnetic spectrum encompassing a first wavelength, λ₁. Filterelement 514 is designed to selectively pass a bandwidth portion of theelectromagnetic spectrum encompassing a second wavelength, λ₂. In anillustrative mode of practice, the wavelengths λ₁ and λ₂ could beselected so that a target substance of interest has a characteristicspectrum such that the ratio and/or other attribute(s) of its spectrumat these two wavelengths distinguishes the target substance from one ormore other background substances. In other modes of practice, the valueof such ratio might be indicative of the concentration of a targetsubstance in a liquid medium being imaged. System 500 further includes areflector module 518 having reflector faces 520 and 522.

In use, system 500 is used to capture filtered images of a scene (notshown) to detect the presence, location, and/or quantity of a targetsubstance in the scene. Incoming light 524 is captured and filtered byfilter elements 512 and 514 in filter card 510. Filter element 512selectively passes a narrow bandwidth portion 526 of the incoming light524. Bandwidth portion 526 in illustrative embodiments has a bandwidthof up to 20 nm, or even up to 15 nm, or even up to 10 nm, or even up to5 nm or less. Preferably, λ_(i) is located substantially in the middleof this bandwidth. Filter element 514 passes a similar bandwidth portion528 with respect to λ₂. Image capture element 502 captures a firstfiltered image for the filtered bandwidth portion 526. Image captureelement 503 similarly and substantially simultaneously captures a secondfiltered image for the filtered bandwidth portion 528. System 500includes program instructions that analyze the captured imageinformation to determine if the attributes of the image information atλ₁ and λ₂ is indicative of the presence, location, and/or quantity ofthe target substance at one or more locations in the scene being imaged.

FIG. 22A shows how system 500 can be modified to capture greater numbersof filtered images to provide modified system 600. System 600 includesan image capture array including three image capture elements 602.Filter card 604 includes a spectral filter array including three filterelements 606. Reflector component 608 includes three reflecting faces610 to reflect, respectively, filtered light to corresponding imagecapture elements 602. Another modification is shown as system 700 inFIG. 22B. System 700 includes an image capture array including fourimage capture elements 702. Filter card 704 includes a spectral filterarray including four filter elements 706. Reflector component 708includes four reflecting faces 710 to reflect, respectively, filteredlight to corresponding image capture elements 702.

FIG. 23 shows another embodiment of a spectral imaging system 800including an image capture array including six image capture elements802 arranged in two generally linear sub arrays on each side of areflector component 808. Reflector component 808 includes two elongatereflector faces 810 that span the lengths of the adjacent arrays ofimage capture elements 802. Filter card 804 includes a spectral filterarray including six spectral filter elements 806. Each spectral filterelement filters a pre-selected bandwidth portion of the incoming light801 to produce corresponding filtered light 812, 814, 816, 818, 820, and822. In each case, reflector 808 redirects the filtered light 812, 814,816, 818, 820, and 822 to a corresponding image capture element 802.Each image capture element 802 thus captures a corresponding filteredimage. System 800 includes program instructions that analyze thecaptured image information to determine if the characteristics of theimage information at the captured wavelengths is indicative of thepresence, location, and/or quantity of a target substance of interest atone or more locations in the scene being imaged.

FIG. 24 shows another embodiment of a spectral imaging system 900including an image capture array including first and second imagecapture elements 902 and 903. Filter card 904 includes a spectral filterarray including first and second spectral filters 906 and 907. Filter906 has a bandwidth to selectively pass light including λ₁. Filter 907has a bandwidth to selectively pass light including λ₂. Reflectorelements 908 and 909 include corresponding off-axis, parabolic mirrors914 and 915 to reflect and focus light onto a corresponding imagecapture element 902 or 903. Use of such parabolic reflecting surfaceshelps to focus different wavelengths to a common focus. This is onestrategy to reduce chromatic aberration effects for differentwavelengths. The use of separate image capture elements 902 and 903 alsoallows each image capture element to optimally focus on a narrowbandwidth of filtered light. This is an additional strategy to helpreduce chromatic aberration effects for systems which use a lens as partof the focusing mechanism. All embodiments of the present invention thatprovide independent focusing of image capture elements share thisadvantage.

In use, incoming light 910 is filtered by filters 906 and 907,respectively, to provide first filtered light 912 encompassing λ₁ thatis parabolically reflected and focused toward image capture element 902and second filtered light 913 encompassing λ₂ that is parabolicallyreflected and focused toward image capture element 903. Image captureelement 902 captures filtered light 912, while image capture elementsubstantially simultaneously captures filtered light 913. System 900includes program instructions that analyze the captured imageinformation to determine if the characteristics of the image informationat the captured wavelengths is indicative of the presence, location,and/or quantity of a target substance of interest at one or morelocations in the scene being imaged.

The present invention will now be further described with reference tothe following illustrative examples.

Example 1 Detection of Water

Referring to FIGS. 25A to 25E, the principles of the present inventionare illustrated by an example having two glass vessels partially filledwith water that are within the field of view of a spectral imagingsystem of the present invention. The system is used to detect thepresence of water in the two vessels. In each of FIGS. 25A through 25E,the vessel on the left is a round drinking glass. The vessel on theright is a square glass flower vase with a bottom of non-uniformthickness. It is desired to generate a display image that shows theposition of the water within each vessel, as well as an indication ofthe spatial position of objects within the field of view. FIG. 25A is a‘full spectrum’ image taken with the narrower visible spectrum filterremoved (typically included with digital cameras); FIG. 25B is an NIRdetection image, sensitive to a narrow bandwidth centered on theabsorption peak of water at 980 nm. FIG. 25C is a spectral referenceimage centered in an area of the spectrum which exhibits little spectralabsorption for water at 766 nm. FIG. 25D is a ratio image representingthe reference image divided by the NIR image. FIG. 25E is a displayimage created from the reference image, those regions of the ratio imagewhere the water is present, and a custom colormap to highlight theposition of water. A reference spectrum for water is shown in FIG. 2A.

In order to detect the water within the two vessels, it is firstnecessary to examine the spectrum of water. FIG. 2A shows the absorptionspectrum for water in the visible and near-infrared regions. Waterexhibits a large absorption peak near 980 nm with minimal absorptionelsewhere in this spectral region. Therefore, two wavelengths areselected to distinguish water from other background, namely 980 nm atthe absorption peak and 766 nm, in a region of the spectrum with minimalabsorption.

A detection image is assigned to the wavelength of 980 nm in order tocapture the absorption peak and a reference image is assigned to 766 nm,since minimal absorption is evident at this wavelength. Since theassigned wavelength of the reference image exhibits little absorption,and the reference wavelength is close to the visible spectrum, thereference image at 766 nm will also be used as the spatial referenceimage. Based on these assignments, a multi-camera array with twoelements is used to capture filtered images corresponding to each of thetwo wavelengths. The multi-filter array uses two filter elements,namely, 980 nm and 766 nm.

Elements of the filter array and the multi-camera array were opticallyaligned in order to acquire the set of images. While numerous automatedalignment methods are available, these two example images were manuallyaligned by inspection. Both the detection image and the reference imagewere normalized for illumination based upon the intensity of the sheetof paper beneath both vessels. This normalization accounts for filtercalibration, illumination normalization, and sensor calibration since itis based on the intensity of an equivalent in-frame reference withuniform spectral properties at the two wavelengths of interest.

While sophisticated, multivariate approaches exist to detect theinformation regarding the target substance, the presence of water wasdetermined in this example by computing the ratio of the detection imageto the reference image. Regions of the resulting image above a giventhreshold were determined to be the pixels associated with the presenceof water. More specifically, the values of the reference image weredivided by the corresponding values of the NIR detection image with theresult as shown in FIG. 25D. Because the pixels associated with waterare darker (lower in numeric value) in the detection image, FIG. 25B,the resulting ratio is larger for those pixels representing water, FIG.25D.

A display image (FIG. 25E) was computed from the spatial orientationimage (also the reference image in this example) and the ratio image.First, the display image was assigned the exact values of the spatialorientation image. This will permit those regions of the field of viewthat do not contain water to be seen as they appear in the visiblespectrum. Then the ratio image was interrogated to identify those pixelswith values greater than the threshold value, namely, those pixelsvisible in FIG. 25D representing water. Next the values of the ratioimage greater than the threshold are transported to the display image intheir corresponding positions, the result being a display image havingthe exact values of the spatial orientation image where there is nowater present and a display image having values equivalent to the ratioimage where there is water present.

The final step in creating the desired display image is to create acustom colormap that highlights the presence of water in the displayimage, FIG. 25E. The colormap, as implemented in FIG. 26, includes amatrix that is typically 256 rows by 3 columns. The columns representthe colors red, green and blue respectively. The rows represent thevalues for each color in its respective column. FIG. 26 shows such acustom colormap. For most of the range of values within the displayimage, the red, green and blue values are equal resulting in a grayscaleimage. For values near the top of the display range, the blue column iszero with equal red and green values. Equal red and green values, withno blue, result in yellow.

The slope on the yellow segment of the colormap, as shown, is relativelyshallow. This implementation shows little change in color intensity forvarying values within the image. As shown, this colormap is best forindicating the presence of the target substance. If it is desirable toindicate a quantity of the target substance, the slope of the red-greenline may be increased to provide a greater variation in color intensityover the same range of image values. The selection of columncombinations provides a vast color palate to indicate any number oftarget parameters.

FIG. 27 shows another way to generate an output image based on theimages obtained in FIGS. 25A to 25E. The water presence is highlighted,e.g., shown as yellow or shaded as shown, while the non-water region ofthe image is a grayscale representation of the spatial image showing thespatial position of objects that contain no water. A custom colormap isapplied to a three-dimensional graph of the display image, FIG. 25E. Thehigher values, represented with highlighted, e.g., highlighted withyellow or shading as shown, were inserted into the image from the ratioimage, representing the presence of water.

Example 2 Hyperspectral Imaging for Obtaining Reference SpectralInformation

This example shows how spectral information determined by hyperspectralstudies may be useful to the present invention in helping to select thecenter wavelengths of the filter array elements for detection of aparticular target substance.

This study was performed with respect to durum wheat, triticale, andbarley. FIG. 28A shows spectra obtained with FOSS instruments showingthe position of the selected wavelengths on the average spectra for thethree grain types. From these spectra, it is determined that spectralcharacteristics of barley at 1110 and 1470 nm could be used todistinguish barley from the other two grains. FIG. 28B shows a customcolormap applied to the ratio values at these two wavelengths. FIG. 28Cshows the resulting spatial image with detected areas indicating theratio of the selected spectral amplitudes. The top row of imagescorresponds to wheat, the middle row to barley, and the bottom row totriticale. Barley may be clearly distinguished from the other two grainsdue to the lower ratio, while durum and triticale kernels are shown tobe much closer in chemical composition. This is expected since triticaleis a hybrid of durum wheat.

In the practice of the present invention, spectral imaging can thereforebe used to distinguish barley from the other two grains. A multi-cameraarray would capture at least first and second images of the grains ofinterest through spectral filters allowing a first image to be capturedfor 1110 nm and a second image to be captured for 1470 nm. If desired, athird spatial reference image can be captured. The ratio of the spectralresponse at each of the two wavelengths is computed. Pixels in which theratio is below a threshold are assigned to the background, pixels withthe ratio in a first range may be assigned to barley, while pixels witha ratio in a higher range may be assigned to durum and triticale. FIG.28C shows the display image capability with this kind of analysis.

It is worth noting that an algorithm developed for a conventionalhyperspectral imaging system, requiring a first or second derivative ofthe continuous spectrum, may not be best suited for implementation inthe present invention. Such hyperspectral derivative computations,seeking to detect a rapidly changing slope (first derivative) or rate ofslope change (second derivative), typically depend on a large number ofcontiguous spectral samples. The present invention focuses on a lowernumber of discrete, non-contiguous spectral samples. However, if a keyspectral characteristic happens to be a rapidly changing spectral slope,selected wavelengths of the present invention may be strategicallyplaced along the desired slope in order to detect the necessarycharacteristic. In general, slope calculations are susceptible to noisecontamination in the data. Therefore, the strengths of the presentinvention, namely, to seek spectral levels rather than slopecalculations, provides a preference for reliable data analysis—which isan advantageous signal processing strategy for even hyperspectralsystems.

Example 3 Egg Inspection

As an example of the flexibility of the present invention, egg spectraldata of U.S. Pat. No. 4,182,571 (issued 1980) are used to select filterwavelengths and determine an algorithm to discriminate between good anddefective eggs. While U.S. Pat. No. 4,182,571 teaches an optical systemto obtain a single measurement per egg using three wavelengths, thepresent invention permits a high resolution image of each egg using twowavelengths implemented in the fashion of the present invention, namely,two filter elements on a filter card processing a similar number of highresolution images. Wavelengths of 567 nm and 610 nm are selected basedon the spectral information. In this embodiment, the present inventionprovides an analysis for each pixel in the egg image. The image can beanalyzed at lesser resolution if desired.

This example shows that practice of the present invention may be used todiscriminate between good and defective eggs. As compared to U.S. Pat.No. 4,182,571, the present invention accomplishes this with improvedprecision due at least in part to the greater resolution achieved by theimage capture methodology of the present invention. Additionally, thepresent invention, by analyzing each pixel of a high-definition image,have used the difference between blood detection and other spectra toimage the vasculature within an egg. This allows the monitoring andmeasurement of fertilized egg development.

The spectral data for four egg conditions are shown in FIG. 29A. Theseare candling transmission spectra for white shell eggs including a newegg, old egg, addled egg, and bloody egg, redrawn after data presentedin U.S. Pat. No. 4,182,571. The processing illustrated in FIG. 1 isapplied to these spectral data.

The exemplary methodology of FIG. 30 was used to create a targetalgorithm to identify different egg conditions. In step (a), a scattercoefficient was computed as a function of wavelength (not shown). Thisscatter coefficient was used to help determine an optimal selection ofwavelengths in order to distinguish one egg condition from another usingthe egg spectral information shown in FIG. 29A. The scatter coefficientis a measure of the variation or dispersion of these spectral data ateach wavelength. This representative formula serves as an example ofnumerous measures of dispersion. Alternative examples include standarddeviation and variance. The first wavelength, λ₁, to be used in theanalysis was selected to coincide with the peak in the scattercoefficient function in step (b), since at this wavelength thedifferences among spectra are greatest. In step (c), the positive tonegative zero crossing of the first derivative of each spectrum (shownin FIG. 29B) was used to locate the wavelengths corresponding topositive peaks in each spectra at a predetermined distance from thefirst selected wavelength. The second selected wavelength, λ₂, wascomputed to be the average of these positive peak wavelengths. Step (d)in FIG. 30 was a computation of each spectral amplitude at each of theselected wavelengths. Step (e) involved plotting the spectral amplitudefor λ₁ as a function of λ₂. From this plot, non-overlappingclassification zones may be established as indicated in step (f) andgraphed in FIG. 29C. Future data which falls within these classificationzones may then classified as one of the predetermined classes of eggconditions, namely, new egg, old egg, addled egg, or bloody egg for thisexample.

While this example includes data from only two wavelengths, resulting ina 2-dimensional classification space, the present invention allows theselection of n wavelengths for analysis, if desired, resulting in ann-dimensional classification space. Similarly, the scope of the presentinvention includes both the circular classification zones shown in FIG.29C, and the expanded n-dimensional classification zones for the case ofn selected wavelengths. FIG. 29C is a plot of spectral amplitudes foreach egg condition with the spectral amplitude at 610 nm as the x-axisand the spectral amplitude at 567 nm as the y-axis. Note from FIG. 29Cthat the new egg data is a substantial distance from the other eggconditions, a characteristic that makes new, good eggs relatively easyto distinguish from defective eggs. Additionally, the present inventionincludes classification zone boundaries that are both symmetric andnon-symmetric shapes in n-dimensional space that permit an accurateclassification of the spectra based on data from the spectral amplitudesat the selected wavelengths.

The present invention also would include the use of numerous additionalclassification algorithms and methods, known to those skilled in theart, to determine the selected wavelengths to be implemented in thefilter array associated with the multi-camera array of the presentinvention.

Example 4 Vascular Imaging

The present invention was used for vascular imaging. Arteries and veinscontain blood with different levels of oxygen attached to the hemoglobinmolecules of the red blood cells. A spectral imaging system capable ofdetecting the different spectral properties of oxyhemoglobin (HbO₂) anddeoxyhemoglobin (Hb), as shown in FIG. 31, provides an output imageindicating vascular structure by determining which pixels within atarget field of view fall on arteries and which fall on veins,Additionally, an analysis of the spectra permits an indication of oxygensaturation within viewed tissue based on an analysis of these spectra.

In one aspect, the present invention relates to a method to deriveinformation about arterial blood within the target field of view. Thedetection of arterial blood, and thus the presence of an arterialvessel, may be accomplished by the present invention using eitherreflectance or absorption methods via an analysis of the reflectance orabsorption spectra for oxyhemoglobin and deoxyhemoglobin. An analysis ofthe spectra in FIG. 31 was used to select the optimal wavelengths todistinguish an artery from a vein or background tissue based on oxygensaturation. A spatial, point source example of this concept has longbeen used to noninvasively measure the oxygen saturation of a patient'sblood via pulse oximetry devices. In these devices two LED lightsources, a red source near 660 nm and a near-infrared source in therange of 905-940 nm are typically used. The ratios of transmittedillumination at these two wavelengths at the appropriate times of thecardiac cycle are used to provide an indication of oxygen saturation foreach pixel. The accumulation of the resulting pixels indicating arterialblood is an image showing the arterial vascular structure.

In another aspect, the present invention relates to a method to deriveinformation about arterial and venous blood within a target field ofview by implementing the present invention to obtain images throughspectral filter elements sensitive to wavelengths in the red and NIR,such as 660 nm and 940 nm. By examining the spectral amplitude at two ormore selected wavelengths, based on the spectral characteristics ofoxyhemoglobin and deoxyhemoglobin in venous blood, the spatial positionof venous blood was determined within the field of view. Furthercharacteristics of venous blood may be determined by examining thespectra of those spatial positions identified as venous blood. Acumulative result of pixels having venous blood provides an imageshowing the venous vascular structure.

In another aspect, the present invention relates to a method ofdistinguishing between arteries and veins within the field of view. Byknowing the spectral differences between oxyhemoglobin anddeoxyhemoglobin it is possible to identify and label, via color-codedoverlays on a spatial reference image, the spatial position of arteriesand veins within the field of view. FIG. 17 illustrates this principlefor a retinal image with a single wavelength. A multiple wavelengthembodiment of the present invention will provide even greaterdifferentiation between arteries and veins.

In another aspect, the present invention provides the capability tocompute an oxygen saturation image of tissue within the target field ofview. Pulse oximetry devices have long been used to provide anoninvasive, point source measurement of the oxygen saturation of apatient's blood. In these devices two LED light sources, a red sourcenear 660 nm and a near-infrared source in the range of 905-940 nm aretypically used. The absorption spectra are shown in FIG. 31. The ratiosof transmitted illumination at these two wavelengths at the appropriatetimes of the cardiac cycle are used to compute an indication of oxygensaturation. Similar “point source” measurements for each pixel withinthe target field of view may be made using the present invention. Thecumulative result of these pixel measurements is an image of oxygensaturation for tissue throughout the target field of view.

Example 5 Sanitation

Sanitation of surfaces is a constant challenge in many industries,including the food service industry, the food processing industry, thepharmaceutical industry, the nutraceutical industry, hospitals, clinics,veterinary facilities, and the like. The present invention provides avaluable tool to quickly identify the presence of a contaminant withinthe target field of view.

Generally, in the context of this example, sanitation involvesprotecting against contamination. This is distinguished from anotheraspect of sanitation, which involves the collection and disposal oftrash. The present invention provides a powerful tool that can bebeneficial to help provide sanitation in an effective and economicalmanner.

As an example, food processing equipment must be carefully cleaned atregular intervals. After cleaning, each surface of the processingequipment must be carefully inspected to be sure that no contaminants,such as organic particles, biofilms, or cleaning fluid, remain. Thesecontaminants may not be readily visible to the naked eye. The presentinvention may be used to capture filtered images of surfaces at aplurality of suitable wavelengths, use those images to detect thepresence and location of contamination, and then to display eachcontaminant as a color-coded region on an image of the surface of thefood processing equipment. The present invention allows this detectionand locating to occur with very high resolution. Such an application forthe poultry industry is shown in FIG. 18.

In one aspect, the present invention is related to a method to detectand locate contaminants on a surface which may come into contact withfood for human or animal consumption.

In another aspect, the present invention is related to a method todetect and locate contaminants on a surface where the role of targetsubstance and background substance are intentionally reversed. In thisaspect the common background surface is identified and the contaminant,in turn, is any substance which is not the background surface. As anexample, in the food industry (and many other industries) stainlesssteel is used as a surface material for countertops and equipmentsurfaces. In this aspect, the common spectrum of stainless steel is usedas the target substance. Any surface area which is not stainless steelis then identified as the contaminant.

In another aspect, the present invention is related to a method todetect and visualize contaminants on any surface which may come intocontact with food for human or animal consumption, such as but notlimited to, food processing equipment, food preparation surfaces,cooking surfaces, cooking equipment, countertops, vats, beveragecontainers, storage containers, fermentation containers, stirring vats,pipes, conveyor belts, test equipment, inspection equipment, meatprocessing equipment, ovens, utensils, vacuum equipment, and bottlingequipment.

In another aspect, the present invention is related to a method todetect and visualize contaminants on any food packaging surface whichmay come into contact with food for human or animal consumption, such asready to eat packaging, aseptic processing, plastic trays, bags, boxes,cans, cartons, flexible packaging, pallets, and wrappers.

The methods used to detect contaminants on food surfaces may also beused to detect contaminants on the surfaces within food transportationcontainers or vehicles.

In one aspect, the present invention is related to a method to detectcontaminants in any number of transportation containers or vehicles,such as boxcars, trucks, ships, and shipping containers using themethods described herein.

In another aspect, the present invention is related to a method todetect contaminants in transportation containers or vehicles whichtransport food animals such as hogs, poultry, cattle, and dairy cows.

In another aspect, the present invention is related to a method todetect contaminants on any surface where the spectrum of the surface andthe spectrum of the contaminant may be distinguished in a mannerdescribed within this specification.

Example 6 Road Conditions

Variations in the condition of road surfaces due to weather conditionsare a well-known hazard in many parts of the country. The spectra(acquired by researcher Tom Ulrich Quantifying Spectral Diversity withina MODIS Footprint—Goetz Recipient Research in the Himalayas,http://discover.asdi.com/bid/93042/Quantifying-Spectral-Diversity-within-a-MODIS-Footprint-Goetz-Recipient-Research-in-the-Himalayas)corresponding to each of these conditions is unique and readilydistinguished one from the other. Specifically, the spectra emitted bydry roads, slush, snow, ice, and water are different as shown in FIG.32A. Thus, the present invention is capable of remotely andnoninvasively capturing filtered images of a road scene via a trafficcamera. The spectral imaging principles of the invention then can beused to accurately identify variations in road conditions, such as snow,ice, slush, wet, salt, hail, mud, gravel, sand, broken glass, and dry asshown in FIG. 32B. Additionally, the display images may be distributedto appropriate personnel and/or drivers via the internet or othercommunications network.

With regular vision, the portions of a road that are merely wet ratherthan icy are not easily identified. By capturing filtered images of theroad at a plurality of suitable wavelengths, an output image of the roadcan be generated that clearly shows and distinguishes dry, wet, icy, andsnow conditions in the same scene. In another aspect, the presentinvention relates to a method to detect variations in road surfaceconditions due to weather, such as snow, ice, slush, wet and dry road asviewed from an automobile with the camera system positioned within orupon the vehicle. In this aspect the detected road condition may beindicated on an output display in a color-coded manner such as isindicated in FIG. 33. The display may be presented to the driver of thevehicle as well as distributed to other drivers via the internet orother communications network. The output can be displayed on a screen ina vehicle or onto a surface of the a windshield or the like using headsup display technology. Driving safety is significantly enhanced when icylocations are pinpointed like this with high precision.

Example 7 Liquid Classification

Each year millions of airline passengers in the United States must limittheir carry-on liquids to volumes in order to comply with TSA securityregulations. These regulations were established to reduce the chancesthat dangerous liquids could be smuggled onto an aircraft. The presentinvention may be used to classify and identify the actual content of theliquids providing greater safety or perhaps providing convenience topassengers by permitting useful volumes of safe liquids. The principlesof the present invention may be applied to liquid classification inthese and other circumstances.

FIGS. 34A to 34D schematically show how liquid classification may bepracticed using principles of the present invention. FIG. 34A shows avisible color image (VIS-COLOR, 400-700 nm) of a scene including severalliquids held in various containers. FIGS. 34B, 34C, and 34D respectivelyshow a reference image (REF, 670 nm), an ultraviolet image (UV, 365 nm),and a near infrared image (NIR, 980 nm). The bottled liquids were water(W), urine (U), motor oil (M), brake fluid (B), and Seven-Up (7). Theletter code and corresponding liquid also are listed in the key in FIG.34B.

The classification chart of FIG. 34D shows one illustrativeclassification that is possible using the qualification of ‘light’ (0)or ‘dark’ (1) with respect to the appearance of the liquid due to theabsorbance associated with the NIR and UV wavelengths. This spectralimaging embodiment easily allows the liquids to be classified into threecategories based on NIR, UV and Reference absorption: 1-0-0 liquids(water and 7-up), 1-1-0 liquids (urine), and 0 1 0 petroleum-basedliquids (motor oil and brake fluid). Such a system, adapted for otherliquids could be applied to classify and screen carry-on liquids forairline security.

In another aspect, the present invention relates to a method to classifyand identify liquids in a container using the methods described herein.

Example 8 Sunscreen

This example shows how the present invention can be used to locate wherea liquid, dispersion, gel or the like is distributed in a scene. Forpurposes of illustrating this aspect of the invention, this exampleshows how spectral imaging can be used to evaluate the extent to whichsunscreen is applied to a person. This example also shows how the liquidcan be easily detected and located in a scene. The uses of such anembodiment include, but are not limited to, ensuring a completeapplication of the sunscreen product to all exposed skin areas, aconfirmation after a period of time that the applied area is stillcovered, and the detection of portions of the applied area where thecoverage has degraded, perhaps due to perspiration or water exposure,indicating the need to reapply the sunscreen product. In other similarmodes of practice, this approach can be used to assess the degree towhich a varnish, powder coating, primer, paint, or other coating isapplied onto a substrate.

In practice, a reference image may be a visible spectrum image displayedin grayscale. Detection results can then be highlighted on such animage. The distribution of sunscreen on the skin of the subject shows upvery dark when viewed with a spectral filter element that is sensitiveto a wavelength in the ultraviolet region of the spectrum. FIG. 35Ashows an image of a plate on which sunscreen has been placed. The lefthalf contains Coppertone Sport, SPF 30. The right half contains BananaBoat Lip, SPF 50. In this image, both brands of sunscreen appear to beinvisible. Viewed under visible light, the locations of the sunscreenare not easily discernible. FIG. 35B shows an image of the same platecaptured with a spectral filter element that is sensitive to awavelength in the ultraviolet region of the spectrum. An image showingthe sunscreen using a filter element passing only an ultravioletwavelength which shows the absorption of the UV light where thesunscreen is smeared as a very dark region for both sunscreen products.In contrast, the visible spectrum image of FIG. 35A provides little ifany visual feedback when applying an invisible sunscreen product to theskin. While the two brands appear similar, they do have differentspectral properties. If it was desired to detect and identify thedifferent brands, images at additional, suitable wavelengths could becaptured that would allow such differentiation in the images.

FIG. 36A shows a simulated image in the visible spectrum in whichsunscreen is placed on skin areas of a child. The locations of sunscreenon the skin are not visible. FIG. 36B shows a simulated image in whichan image of the child is captured using a filter element sensitive to UVlight. The areas of skin covered by sunscreen are now easily detectedand shown as darker areas. As shown, there are areas of skin which areadequately covered with sunscreen and there are areas which have beenmissed, leaving the skin susceptible to sunburn. Gaps in coverage arevisible around the eyes, on the underside of the forearm, on the hand,and around the neckline.

FIG. 37 shows an additional embodiment where at least one additionalcamera and filter element are used to detect both the regions of theskin covered by the sunscreen as well as the regions of the skin thatare not covered with sunscreen.

The image of FIG. 37 is generated by capturing further image informationusing an additional wavelength in addition to the single wavelengthevaluation used to obtain the image of FIG. 36B. With the addition of asecond wavelength, uncoated skin also is detected so that both coveredand uncovered skin can be identified with respective indicia. In actualpractice, the output image can be color-coded to show the application ofsunscreen in blue and the uncovered skin in red, representing thepotential for sunburn. The color overlays may be applied at a reducedopacity to allow features to be seen through the overlays.

FIGS. 38A, 38B, and 38C show how easy it is to locate a liquid orsurface coating in a scene where visual inspection would be very timeconsuming or even hopeless. In FIG. 38A, a surface coated with sunscreenis located somewhere in the scene, which is an image captured in thevisual spectrum corresponding to what the naked eye sees. The locationof the coated surface, sunscreen on a person's face, is hidden in thefoliage and difficult if not impossible to discern with the naked eye.For reference, a circle is placed around the region of the image inwhich the sunscreen placed on person's face is located. Even with thishint, the sunscreen and the person are not easily seen.

Using principles of the present invention, FIG. 38B is a close-up outputimage derived from spectrally filtered images that allows the sunscreento be easily detected and located in the scene. The same area bounded bya circle in FIG. 38A is bound by a circle in FIG. 38B. The detectedsunscreen area is highlighted with suitable indicia, such as a bright orcontrasting color. In some embodiments, the detected area is used togenerate the red circle around the sunscreen region. The red circle isthen applied back onto the corresponding location in the base image ofFIG. 38A.

FIG. 38C is a close-up image of the area of interest in FIGS. 38A and38B to show how accurately and easily the principles of the presentinvention make it to locate a coated surface or other substance in anotherwise complex scene.

Example 9 Light Source Identification

The present invention optionally may automatically determine the lightsource in a scene by a comparison of the ratio of light present atselected wavelengths. As an example, FIG. 39 shows the discrimination offour different light sources, namely, sunlight, incandescent, white LED,and compact fluorescent using three wavelength regions, designated as A,B and C. A measurement of light emission at the three wavelengthregions, A, B and C, permits a unique determination of which lightsource is present. The following table associated with FIG. 39 shows theunique relative amplitude of light for each source in the designatedwavelength bands:

Light Source A B C Sunlight 45%  100% 50%  Incandescent 0%  50% 100% White LED 0% 100% 0% Compact Fluorescent 0%  50% 0%

It should also be noted that it is possible to discriminate between thefour sample light sources by using only wavelength bands B and C, apossible reduction in the number of camera/filter elements.

In another aspect, the present invention relates to a method toautomatically determine the light source in a scene by monitoring andcomparing image data from different selected wavelengths using themethods described herein.

All patents, patent applications, and publications cited herein areincorporated herein by reference in their respective entireties for allpurposes. The foregoing detailed description has been given for clarityof understanding only. No unnecessary limitations are to be understoodtherefrom. The invention is not limited to the exact details shown anddescribed, for variations obvious to one skilled in the art will beincluded within the invention defined by the claims.

What is claimed is:
 1. A method of using a spectral imaging system tocapture image information to detect one or more target substances in ascene, comprising the steps of: a) providing an image capture arraycomprising a plurality of image capturing elements; b) providing aplurality of interchangeable, spectral filter arrays that areinterchangeably selectable to be optically aligned with the imagecapture array, and wherein each spectral filter array comprises aplurality of spectral filter elements and a filter array identifier; c)selecting and optically aligning a spectral filter array with the imagecapture array to provide a plurality of wavelength channels thatindependently capture spectrally filtered images; d) using the filterarray identifier to automatically identify the selected spectral filterarray; e) automatically selecting one or more algorithms pre-associatedwith the identified spectral filter array from a library of algorithms;f) using the wavelength channels to capture spectrally filtered imageinformation of the scene; and g) using the captured spectrally filteredimage information and the automatically selected one or more algorithmsto determine whether the one or more target substances are detected inthe scene.
 2. The method of claim 1, wherein step e) comprises using anidentifying characteristic on a filter array to select an associatedalgorithm.
 3. The method of claim 1, wherein the plurality ofinterchangeable, spectral filter arrays comprises a plurality ofspectral filter cards, wherein each spectral filter card comprises afilter element array comprising a plurality of filter elements.
 4. Themethod of claim 1, wherein the filter array identifier comprises an RFIDtag.
 5. The method of claim 1, wherein the filter array identifier iselectronically readable.
 6. The method of claim 1, wherein step e)comprises using an orientation of a filter array in the system to selecta pre-associated algorithm.
 7. The method of claim 1, wherein step e)further comprises automatically selecting a user interface associatedwith the identified spectral filter array that is used to detect atarget substance in the scene.
 8. The method of claim 1, wherein analgorithm that is pre-associated with an identified spectral filterarray analyzes captured image information to determine which portion(s)of the image information, if any, have spectral characteristics thatmatch the target substance.
 9. The method of claim 1, wherein eachwavelength channel includes an image capturing element of the imagecapture array and a filter element of the array that is opticallyaligned with the image capturing element.
 10. The method of claim 1,further comprising the step of providing an output indicative of whetherthe one or more target substances are detected in the scene.
 11. Themethod of claim 10, wherein the output indicates a location of the oneor more target substances in the scene.